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FOREWORD 



Citibank's history of contribution to the quality of life In New 
York City is extensive^ and needs no wordy conmient here. Whenj however^ 
early efforts to improve counseling services in the city's high schools 
resulted in data bearing upon the need for Information services to stu- 
dents^ Citibank undertook to sponsor a project of considerable magnitude 
and direct service to the public schools unequalled by any one agency in 
the private sector* 

The CASE institute for Research and Development In Occupational 
Education (IRDOE) Is proud to have been chosen the project manager of 
the undartaklng-^-a computerlEed information delivery systOT-"and presents 
harewlth the final report of the activity and findings* Of special note 
iti this highly successful dCTdonstration of computer-assisted guidance is 
that it representa the first attanpt at service to high-density^ urban 
populations* With feasibility proven^ there Is every reason to believe 
that the tool of automated information delivery will ultimately free 
counselors to perform the more professional tasks for which their training 
has prepared them and thereby improve the quality of service to students* 



Lee Cohen 
Director^ IRDOE 



The report that follows contains much data--numbers^ percentages^ 
and stattstlcal comparisons. It does not include the nmes of the many^ 
many people who took part in some aspect of the CAG project during its 
duratii^n* In ^ome. sense^ the numbers provide the proof of the great 
effort made hyt 

& The high school admlnistratori * We would like to acknowledge 
the generous and warm cooperation of Mr, Murray A, Cohn^ Prin- 
cipal of Louis D, Brandeis High School^ and Ms. Hannah Lewls^ 
Acting Principal during his leave; Mr. Noel Louis^ Principal 
of Herbert Hp Letnnan High School^ and Mr. Kenneth Tewel^ Assis- 
tant Principal; Mr. Leonard J. Harrison^ Principal of Midwood 
High School; Mr, Melvtn Serisky^ Principal of Francis Lewis 
High School^ and Ms, Pauline Welnstein^ Assistant Principal; 
Mr. Herbert Balish^ Principal of Port Richmond High School^ 
and Mr. Barnard C, Fettanan and Ms, Ethel Van Horne^ Assistant 
Principals, 

# The site liaisons n It is with deep feelings of respect and 
friendship that we tharik the five primary liaisons to whom 
this report Is dedicated: Ms. Tina Houck (Brandais)^ Mr. Ron 
Kutseher (Leteian)^ Ms. Mauraan McGinniss (Midwood)^ Ms* Marlene 
Buckley (Francis Lewis)^ and Ms. Alice Farkouh (Port Richmond), 
Without their interest and high ideals^ the CAG project would 
not have been as profitable an experience. 

o The professional and administrative staff of the five high 
schools who worked with us. At Brandeis-*-*we would like to 
thanfc Mr* Danny Grossman^ co^liaison; Mr. Jinmie E. Warren^j 
Mr. Peter Steinberg^ Ms. Rebecca Dall, and Ms. Nancy Moccaldi 
of the College Office^ and Ms* Florence Shapiro and Ms. Ethel 
Elkin. At Midwood — ^we acknowledge the work done by Ms, Barbara 
Venito^ co-liaison) and Dr* Sollis Schu^b and Ms. Lillian 
Sapirstein of the College Office; and the help provided by 
Ms* Sybil Del Gaudlo. At Francis Lewis— the efforts of Ms, 
Mollie Levine and Ms* Lillian Pollack are gratefully noted. 
At Port RicImond--we publicly thank Ms. Ethel Bergman and 
Ms. Marlon Johnson of the College Office^ Mr. John Gino^ 
Librarian^ and Ms* Cathy McGovern. 

m The Division of High Schools of the Board of Education of the 
City of New York* In addition to in-kind support of the CAG 
Project^ Mr. S^iel Polatnick^ Director^ and Dr, Filmore Pelg 
and Mr, Arthur Auerbach, provided leadership and support which 
was invaluable. We were encouraged by their interest and help- 
fulness during the darker days* 

m The ITOOE project staff. The patience and perseverance of the 
project staff was noteworthy. As a teaflj we learned from one 
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another and shared our ideas^ responsibility^ and work. We 
tested, obsarved, scheduled, tallied and re-tallied^ counted 
and totalled, delivered rolls of paper, trained staff and 
students, analyEed and interpreted the results, and wrote 
several reports describing our progress and probl^s. 
Mr. Arnold Jaffe, who worked with the project teaai from its 
earliest days through the final year when he took over the 
day-to-day operation; and Ms* Pearl Beck, Mr* Larry Killian, 
and Mr* Joseph Perlman who also assisted with all the aspects 
and contributed to all the decisions e 

0 IRDOE Staff and consultanta. tople and complete backup was 
provided by several staff members and consultants^ whose con- 
tributions were over and above that expected: Mr* Russell 
Nutter, project administrator whose duties ranged from secre- 
tarial to editorial and production activities; Ms* Valentine 
Michielini, IRDOE secretary, who was always there just before 
she was needed; Ms* Melanle Bentley, who often volunteered to 
help with the more repetitive tasks; Dr* Rita Senf, whose 
statistical and editorial talents were so valuable, 

0 CASE, IRDOE Directors* Dr. Lee Cohen, Director of IRDOE, 
maintained a high level of tnterest and commitment to the 
project goals and permitted us the latitude to pursue thKn. 
Dr. Max Weiner, Director of CASE, fiercely and loyally sup- 
ported our research interests. Both Drs. Cohen and Welner 
pravlded the necessary critical climate as well as the 
resources of the Graduate School and University Center at 
those times when the need was ^st crucial. 

a First National City Bank, Urban Affairs Division. The proj- 
ect was Initiated and sponsored by Citibank. Support and 
feedback from Dr. Henry Brenner, Mr. Wlllian C. Herbster, and 
Mr, Herman Dia^ was always available and helpful. We would 
also like to thank Dr^ Norman Wlllard and Mr. Jack Starr for 
their early involvement* Working with Citibank was a personal- 
ly and professionally stimulating experience. 

# Time Share Coroporatlon, The attentive concern of Mr. Charles 
A, Morrlssey and Mr. Herb Cornell in overseeing delivery of 
all contracted services bodes well for good future relation^ 
ships between the public and private sectors* Their aval la-* 
bility to us and the quality of the services they provided 
warrants our faith in similar endeavors* 

9 The students in the New York City public high schools* Several 
thousand young women and men took our tests, answered our ques-^ 
tions with care, and gave us worthwhile, serious and unsolicited 
suggestions and advice* Ihey are the beneficiaries or victims 
of all our •experimental daronstratlOTS. All good go with youi 

Barbara R. Heller Linda Chltayat 

Principal Investigator Project Associate 



iil 

G 



TABLE OF CONTENTS 



Chapter Title PM§ 

ACKNDWLEDQ^NTS il 

TABLE OF TONTENTS ^ * iV 

LIST OF TABLES • . , * vi 

LIST OF FIGURES vi 

I BACKGROUND ^ • 1 

Introduction 1 

Overview of the project 3 

II PROJECT ENViaONMENT • * * * * . . 6 

Participating High Schools 6 

Initial Student Views on College and Career Planning . . 8 

Main Features of the Guidance Information System ... * 11 

III EVALUATIDN METBDDS * 17 

XV PROJECT n4PI£MENTATI0N * . * 25 

Project Staffing and Administrative Support 25 

Orientation^ Training^ and Allocation of Staff Time . p 28 

Scheduling and Use of Teminal Time , * , . 30 

Days and Mean Hours Used •« 30 

Equipment Malfunctions • 33 

CAG Operation in the Pilot Schools ^ . 36 

Scb^^l A-#s»*»«*****»» 38 

School B»se«»««*»^*******'**** 39 
School C»e*«***««***«********* 
School D«»#. 
* School E#«o**»»**»»«*"»»»«*»*» 

V ANALYSIS OF THE GIS DATA FILES * * 43 

College Files 44 

Occupational File 48 

VI STUDENT USE OF TIE COMPUTERIZED SYSTEM 53 

Users and Uses #,.-#*•* 53 

Multiple Use 57 

Direct Vs. Indirect Use 58 

Searches Vs* Descriptions 61 

Data Files Used * *-*..*«^ 64 

Grade Level of Uf^rs ••••« 66 

VII OUTCOMES- ANALYSIS OF COLUEGE APPLICATIONS 69 

Applications Per Graduate (APGs) ^ • 72 

Outcomes of Applications 75 

Reported Effects of CAG on College and Career Plans . • 79 

7 

iv 



chapter Title 



Page 



VIII OUrmMlS: mPACT OF GAG ON STUDENT USERS * • 
LImitatiDns Concerning the Test Results * ♦ 
Preferred Sources of Information * . * * . 

CDllege Information # . • 

Career Infoitnation , * 

Vocabulary 

Description and Organisation of InforTnation 

College Information 

Career Information , . , 

Decision-Making . # * ^ * * * , 



■ « • 



• • • 



82 
82 
84 
86 
86 
88 
89 
90 
92 
95 



IX 



PARTICIPANTS' REACTIONS AND DISSEMINATION ACTIVITES 

Participants' RBaetions » 

Dissemination Activities ... 



99 
99 
102 



X SUMMARY, CONCLUSIONS, AW RECOtMEOTATIONS 105 

Sumnary' Background and Objectives 105 

Conclusions and Reconmendations . . , , 108 

POSTSCRIPT 127 

CAG, 1975-1976 127 

Project "ACCESS" 128 

APPEHDICHS •••••••■••••■a««a*«a(i»e 131 



8 



V 



LIST OF TABLES 



Number Title Pa^e 

1 Terminal Time Schaduled and Used for Students^ By School and 

By Time Period (Time in Hours^ To Nearast Quarter-Hour) . , 32 

2 Reported Time Lost Due to Equipment Malfunctions in Periods 

1 and 2^ By Month^ All Five Schools Combined 35 

3 Nisnber of Student Users and Total Nuniber of Uses^ By School 
and By Time Period , , , 54 

4 Mean Nimber of Uses Per On-Line Day and Mean Uses Per 
Terminal Hour^ By School and By Time Period 56 

5 Number of Students Using CAG Once and More Than Once^ By 
School^ Across All Time Periods 57 

6 Percentage of Direct and Indirect Use^ By School and By 

Time Period » ^ » m .^^..^ 59 

7 Percentage of Uses Involving Searches^ February 1974 and 
Periods 2 and 3^ By School e»« 62 

8 Total Uses of Each GIS File As a Percentage of Uses Within 
Each Time Period^ By School ^ , 65 

9 Percentage of CAG Users in Grades 12^ 11^ and 10 and 9^ By 
School and By Time Period 67 

10 Nimiber of Graduates and Data on Known College Applications^ 
Including CUNY^ SUNY^ and Other Colleges^ For 1974 (•74) and 
1975 ('75)^ By School . . 73 

11 Outcontes of Non-CUNY Applications 76 

12 Users' and Non-^Users' Opinions of the Three Most Valuable 
Sources of College and Career Informatlonj By School « . « 85 

13 Mean Vocabulary Scores for CAG Users and Non-Users^ By 

S C ho Ol *•■<■••■■••*••••*•■■«■« aaa 88 

14 Mean Scores on College and Career Information Questionnaires^ 
For Users and Non-^Users «• 91 

15 Mean Declslon^Maklng Scores for Users and Non'^Users^ By 

S c ho ol ■■#«■■••■•■••■■■«•■*■«•■■• 96 



9 



vt 



LIST OF FIGURES 



Fl^re Title page 

1 Search Mode: Sanple Interaction Employing Three Basic 

GIS Conroands * 15 

2 Descriptive Mode: Sample of an Actual GIS Description of 

an Occupation * , 16 

3 Instruments for Assessing System Operation^ Use of the 
System^ and Quality of the Data Files 18 

4 Instnmentfi for Obtaining Farticipanta ' Reactions and for 
Asseaament of Students 19 



10 



VI 1 



ERIC 



CHAPTER I 



BACKGROUND 



Introduction 

In Dec^ber 1973^ First National City Bank (Citibank) awarded a 
contract to the CASE Institute for Rasaarch and Development in Occupa- 
tional Education (IRDOE) for its proposal to conduct a demonstration 
and evaluation of a computer-assistad counsallng projecE In New York 
City public high schools. The request for proposals from Citibank 
reflected their long-standing concern with urban educational affairs^ 
as well as the interest and support of the Board of Education's 
Division of High Schools and Bureau of Educational and Vocational 
Guidance with whom they had been associated as members of the New York 
City Guidance Advisory Counuii, The Council^ composed of leaders from 
the public and private sectors^ was formed in 1971 to " broadly assess 
existing guidance services in the public high schools with a view 
toward racoOTiendlng improvements in the practice," As one part of 
their researches^ the Guidance Advisory Council staff was introduced to 
several computer-assisted counseling prograns being triad in other 
school systems around the country i 

In the early 1970s in New York Clty^ the ratio of counselors to 
students averaged approximately l-to-1200^ resulting in "crisis inter- 
vention counseling" which reinforced students' stereotypes that 
counseling and disciplining were fairly synonymous. Both the shortage 
of trained counselors and the steady Increase in the number and complex- 
ity of educational and vocational opportunities for students taxed even 
the moat qualified counselors' time and skill, "ComputerlEed counseling'* 
as an adjunct service^ is a cQmpelling concepts Computers' capacity to 
store^ collate^ retrieve^ and select from a vast store of Information^^ 
and to present the information back rapidly and attractively--enables 
access to more information^ relieves the counselor of the burden of 
collecting and sorting the information^ and thus allows him/her to spend 
more time interacting with the student. Computers could ovlde needed 
Information to many more students^ and even the hypothesis id "imperson- 
ality" of the exchange between the student and the computer mighty under 
certain sets of circumstances^ be considered an asset. Despite the pre - 
coneaived benefits^ all parties agreed that the first computer-assisted 
guidance project In New York City schools should be Introduced with care 
and monitored with concern* 

During the months Citibank was reviewing plans and proposals they a 
so met with several R&D agencies and with vendors of computerized 
progrOTis* Several high school principals^ together with the Director of 
the Division of High Schools^ attended a demonstration of a computer 
system. By the end of 1973^ when the contract was formally awarded to 
IRDOE^ the five pilot high schools that were to participate in the 
demonstration had been selected^ and a computer "guidance" system had 
been chosen e 
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Originally proposed to start ^Ith the beginning of the academic 
year^ Citibank's grant to imOE covered the period from November 1973 
through July 1975* November -Decmber 1973 and January 1974 comprised 
the Planning Phase in ^htch negotiations with the pil6t schools^ in- 
stallation of equipment^ and training of school staff would take place. 
Phase 11^ the operational phases extended from late January-early 
February through' Deoember 1974. During this interval^ students ware to 
have access to the computerized progrm. Priority use of the computer 
between February and June 1974 was established for 11th grade students 
to engage In occupational exploration; the September-DecCTiber 1974 period 
was to be reserved for 12th grade students* use of college information* 
The post-operational phase of the initial contract period^ January-June 
1975^ was to be devoted to data tabulation^ analysis^ and reporting. Due 
entirely to the enthusiasm of the pilot high schools^ Citibank extended 
the operational phase through June 1975 so that students' use of the 
systOT was continued for an additional six Bwnths, 

The extension of the operational phase served another purpose^ per- 
mitting additional study of the effects of the computer experience. 
Prior to this^ IRDOE had two roles--progrm manager and evaluator. In 
our managerial role^ we contracted for hardware and software services^ 
oversaw the installation and maintenance of equipment^ oriented and 
trained school personnel^ ordered and frequently hand -delivered supplies 
and materials^ and perfoOTed a public infermation function* As evalua- 
tors^ we were responsible for monitoring the progress of the project^ 
including maintaining records of the nraiber and types of students who 
made use of the computer^ the procedures the schools established for 
implementing the new project Into existing educational programs^ and so 
On* We also attmpted to ascertain whether the system facilitated col- 
lege applications and whether it Increased the number of students 
receiving counseling. 

The extension of the operation to June 1975 provided us the op- 
portunity to collect data describing the direct Impact of the computer 
experience on student knowledge^ attitude^ and behavior. Such data 
was requested by both the Division of High Schools and Citibank as a 
basis for considering a course of action for the Septembar 1975 through 
June 1976 school year. 

As a result of the impact study findings demonstrating that use of 
the computer benefited students^ both the Board and Citibank decided to 
continue the project^ with modified levels of support^ for the 1975-76 
year. The Division of High Schools of the Board of Education continued 
to absorb the costs of the telephones linking the five teletypewriter 
terminals to the central computer and the allocation of a portion of a 
teaching allotment to each of the pilot schoolsi Prior to the 1975 
school year^ each of the pilot schools had received a teaching unit (a 
one-teacher equivalent) to staff the project; the allotment was halved 
for the 1975-76 year. For this latter period^ Citibank continued to 
finance all computer-related costs^ and reduced their grant to IRDOE to 
cover managerial costs exclusively. 

IMOE^ Center for Advanced Study in Education^ Is part of the 
Graduate School and University Center of the City University of New York. 
During Novanber 1973 through June 1975^ IRDOE TOployed a full-^tuae proj- 
ect director^ project associate^ graduate student asslstcnts^ and other 
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Staff members and consultanta from the University to oversee tiiG 
project activities. The miounts and type of staff changed during the 
courie of the pro ject-^more research assistants were employed during 
peripds in which extensive classroom observations were made^ or when 
testa other instrimaenta were administered^ and when data was analysed. 
Computer specialist i were used during the early part of 1975 when IHDOE^ 

; the Board of Education^ and Cltlbai^ were considering plans to revise 
the computerised information. To manage the. computer-assisted guidance 

' project In 1975-76^ a graduate assistant was able to attend to the ad- 
ministrative detalla and to train school persotmel. The director and 
associate concentrated on policy issues^ continuing to spend a great 
deal of tiiaa In public relations and infomatlon dlssemlnationp 



The authori hope that this report will convey the highly enthusi- 
astic flavor of the reaction of the many people^ at all levels and from 
various backgrounds^ who have seen the project In operation, IHDOE 
staff^ Cltlbaidc^ and the Division of High Schools were Inundated with 
requeata for information^ pleas for Involvonent^ and proposals for ex- 
pansion long before sufficient evidence was available on which to base 
recoamendatioQB for future activltlea. These reactions alone^ apart 
from any data^ suggest a widespread feeling of a need for computerised 
isrvicss in guldance-*related areas « 



Overview of the Project 

The aim of this report Is to suomarlze all our eKperiencea with the 
Gompu tar-As sis ted Guidance (CAG) project during the period from November 

1973 through June 1975^ and to suggest conclusions and reconmendations 
basad on our findings and observations. The five reports submitted at 
regular Intervals during the course of the project were very detailed; 
many of those details^ which were necessarily presented In discrate sag- 
mentSj will not ba repeated hara*^ We wlll^ however^ Include some data 
not previously reported* The Intention now Is to l^ofc at the entire 
experience^ seeking an Integration of findings and a broader perspective 
than was posslbla while the project was In progress* 

This report covers three ttoe periods during which students in five 
high schools had use of the computer syst^i Period 1^ February- June 

1974 (5 months)! Period 2^ Septanber-Decanber 1974 (4 months) • and 
Period 3^ January-June 1975 (6 months). Although the CAG project con- 
tinued to operate In the Bamm schools for another yearj there was no 



The five previous reports on A pCTionstratlon of Gomputer'-Asslsted 
Guidance In New York City High Setooia, with their publication dates^ 
werar "First Quarter Report^" 3-31-74| "Second Quarter Report^" 
6-30-74 I "Third garter Report^" 9-30-74| "Fourth Quarter Report^" 
12-31-74; and "Assessment of Selected Effects of CAG on Student Users^" 
April 1975* Copies cannot be supplied^ but can be inspected at IRDOE* 
The appendices to these reports contained copies of all evaluation 
instnments • 
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further data analyiis after June 1975. We will conment in a final 
poitscript on activitiei and devalopments during the 1975-76 school 
year . 

The basic objtctivee of the GAG project were to demDnstrate the 
feasibility of using an automated systm to provide high school fitudents 
with factual and current information on colleges and careers^ to assess 
the effects of such a syst^oa on student choice^ and its effects on the 
allocation of staff time. We expected that effective use of a computer- 
ized system would increase the number of students eeeking college and 
career infomationj the range of college and career alternatives avail- 
able to thra^ and their awareness of the factors and processes involved 
in making decisions or choices concerning colleges and careers. We 
axpacted alio that there would be a reallocation of counselor time — wlth^ 
for exampie^ lees time spent in routine information-conveying activities 
and more time spent In meaningful interaction with students^ helping them 
match their interests and capabilities with the external requirOTents of 
colleges and occupations. The project concentrated on students in grades 
11 and 12. 

The progran used to test the viability of the concept of computer- 
izing such pildance functions and its applicability to New York City 
students was the "Guidance Infomation Systm" (GIS)^ a product of Time 
Shajis Corporation (TSC)^ a subsidiary of the Houghton Mifflin Company. 
It is a computerized data retrieval syston which provides students or 
staff with Instantaneous access to information about occupations^ two- 
and four-year colleges^ and financial aid. A teletypewriter terminal is 
used to type in requests for Information. Through a telephone linkaga^, 
the request is transmitted to the computer which responds by typing the 
appropriate information from the permanently stored data files. A 
student alane^ but usually assisted by a staff member^ may type in his 
own request or the information may be requested In the student's absencei 
in the latter case^ the printed copy of the computer output must be 
delivered to him. Largely because of fiscal constraints, each one 
of the five high schools had access to the computer about one day a week; 
these accesSj or on-line^ days were scheduled in advance. 

At each school a staff moiiber^ whom we have designated a liaison^ 
supervised the CAG operation; usually the liaison had some paraprofes- 
sional or other assistance. The number of teachers assigned supervisory 
duties differed from school to school and frorajlme period to time period* 
In all instances^ however^ there was continuity In the assigiment of 
liaison responsibility* The liaisons arranged for publicity^ recruitment^ 
orientation^ and scheduling of students for computer use^ and for followup 
after the use. On the day the school had access to the computer the 
liaison usually operated^ or helped operate^ the teletypewriter terminal. 

The five public high schools^ one from each borough, were selected 
collaboratively by the Division of High Schools and the Borough Super- 
intendents. The schools weret Louis D. Brandeis in Manhattan^ Herbert H. 
Letaian In the Bronx^ Midwood In Brooklyn, Francis Lewis in Queens^ and 
Port Rlchnond in Staten Island. ^ 



Throuchout this report the schools will be letter-coded to provide 
anon^lty. i A 
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Although these are all acad^ic-comprehensive high schools^ they 
ware more different than allke^ In both student and school character- 
litics. The student populations differed considarably in ability levels^ 
In the proportions who were college-bound^ in socioeconomic levels^ and 
In ethnic composition- The schools varied in age of the Gchool building^ 
Btzm of student enrolliaent^ mmber of seflslons held^ and rate of utiliza- 
tion (the percent utilization of each building in relation to Its rated 
capacity) . 

Of great importance is the fact that within the coimon framework we 
established^ each school developed unique procedures and routines for 
proiect implmantatlon* The many diversities mong the schools were wel- 
come in this investigation of the feasibility of usi^g computer-assisted 
guidance* While these differences greatly complicate the making of inter- 
school comparison^ at the same time the results furnish a better base for 
generalization about a wider range of organizational atructures and 
taplCTentation procedures* 

One caution to keep in mind while reviewing this report la that the 
particular system used to study computer-assisted guidance (in this case^ 
GIB) may impose Imitations on the generallzablllty of the findings. For 
brevity^ this report generally uses the term "GAG project^" or simply 
"CAG^" but it la actually the "TSG-Guidance-Information-System-Computer- 
Asslsted-Guldance project.'* 
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CHAPTER II 



This chapter provides a picture of the pilot high schools^ as well 
as of the students* perceived needs for college and career information 
before there was much use of CAG* This will sery^e as a general baseline 
against which we can later exmine details of GAG usage. Following this^ 
we will introduce the main features of GIS, 



participating High Schools 

It has been mentioned that the five high schools involved in the 
projact differed on many dlmenaioni* This section will describe these 
differences In some detail. 

Although not a randomly selected sanple^ these schools represented 
the diversity of the New York City public high echoql system* The age 
of the building ranged from School E,tha oldest built in 1927^1 to 
School which was the newest and which did not have a full official 
graduating class until June 1975 (the end of Period 3)* The sl^e of the 
student body and the utilisation rate tended to co-vary; School A was by 
far the largest^ had the highest utilization rate^ and operated from 
about 7*30 a*m, to 4:30 p.m.; School the next largest^ also had a 
high utilisation rate^ and held classes from about 7^30 apm. to 5:30 p.m# 
^ School E was the smallest school^ with an enrollment of approKimately 
one-third of that of School A; it operated the shortest school day^ with 
no overlapping sessions. 

In terms of the percentage of students reading two or more years 
below grade level^ School A was the highest (70%), with the four other . 
schools much lower and much closer together^^C^ and E with 20%^ 16%^ 
and 26% reading two or more years below grade level^ respectively. The 
percentage of graduates (axcluding School B) applying to college was 
related to the reading scores ^ except for School A, The rank order was: 
A (with the most applying)^ and then E with a tmch lower percentage. 

These figures do not agree with students' statOTients about their college 
plans^ as we will describe in Chapter II. In socioeconomic level^ 
Schools C and D were in the most affluent coninunities | the families of 



Board of Education of the City of New York, High School Profiles, 
I97l/1972 » prepared by the Division of Systems Planning and Progr^ 
Analysis/ Office of Plannlng-Progrfflnning-Budgeting. This resource pre- 
senti a statistical overview of the high schools for the 1971-1972 year* 
School B was not yet included^ and docraiented comparative infomation 
Is not available; its student population characteristics are sMiilar to 
those of School E, 
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students in Schools E and B were largely blue collar; and in School A 
they ware the poorestj with many fmilies unemployed and on welfare. In 
ethnic mix^ the proportion of non-white students was about a siKth at 
School and just over a fourth at Schools C and D, Over 90% of the 
School A students were non-white, 

Another school variable was the extent of Involvement In other 
special educational programs; some schools--Aj for example--seemed to be 
involved In innimerable projects^ while School C had only a cooperative 
work-study progrffln. In School E the special attention GAG aLforded to 
studenta and staff was quite a novelty. Two schools (A and D) offered 
courses in data processing; one of these (School D) had its own mini- 
computer and offered instruction in prograimlng. Certain administrative 
functions^ such as students' programs^ had been computerised by the 
Board of Education before the 1973-74 year^ and several schools had had 
ascposure to these kinds of computer fonctions. 

In February 1974^ when GAG started up^ one of the more salient 
school characterlatics was the extent of available college and career 
advisory servlc s* School A maintained a modest college office arid had 
no career advisor; it was a participant in a small career guidance proj- 
ect whiah ended in June* Of all the schools^ B was the tiKsst career- 
oriented; it had a career academy^ offered students career decision- 
making courses^ and was involved in a large state-funded project to 
Improve career education. Since it was a new school with no full 
graduating class^ college advising services were minimal. School 
on the other hand^ had an elaborate college advisory service and 
established systems i there were three college advisors assigned to the 
college office* School C had a licensed guidance counselor assigned to 
work with studants on occupational choices. School D also I a large 
and well-established college office staffed by at least two advisors; 
there was little CTaphasis on career counseling* School E's college and 
career office was staffed by two advisors* In addition^ this school was 
involved in a small special career education program which ended in June 
1974 with the first in a series of New York City fiscal retrenchments. 

Many of these student and school characteristics have direct 
bearing on the results of the GAG project. The proportion of the students 
going on to college and the ability level of the student population af- 
fectad interest in and use of the specific kinds of Information stored In 
the computer^ as well as how much help students would need to understand 
the system. The size of the student body determined the number of 
studenta eligible for service. The number of sessions held affected the 
availability of students* a school operating on nwre than one session 
tends to discourage students from being in the building after the end of 
their school day. Space constraints affected the location of the CAG 
equipment and thus its accessibility. During the time periods studied^ 
economic concerns becane increasingly Important in making college and 
career decisions; they also affected the availability of staff ^ particular- 
ly in the 1974-75 year (Periods 2 and 3), 
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initial Student Views on Cq liege and Career Planning 

In March 1974^ approKtoately one month after the praject becmne 
operationalj we sdminlstered the Survey of Student Activities and Plans 
to 1889 juniors ai^ seniors in the five pilot schools. 1 The Survey 
investigated the students' initial status--their current plans for edu* 
cation and/or occupations after high school^ sources of influence on 
their decisions^ and their perceptions of the college and career informa- 
tion they had received in high school prior to adminiitration of the 
Survey, 

We asked students to indicate what they thought they would do 
Ijmedlately after graduating*- For a large majority of the respondents^ 
high school graduation did not mean the end of their education^ even 
though many eKpected to work while going to college. At Schools C and D 
over 90% of the respondents plaimed to continue their education. 
Schools A and E were intermediate in this respect, while about three- 
quarters of the students tested at School B said they planned to continue 
their education*^ These percentages were Glightly higher for juniors 
than for eenlori^ except at School E. 

Most students planning further education eKpected to start to- 
mediately after high school; about 30% to 407^ expected to work at the 
same tlme^ with Schools A and E being at the higher end of this range. 
Of those expecting to continue schooling Immediately after graduation^ 
most chose a four-year rather than a tvtQ^ymM college. Very few 
students planned to get a job and then go to a school or college lat6r 
on; and few expected to go to a specialised training school (technical^ 
art^ etc). These results suggest that potentially large nirabera of 
students might be interested in Infomation about colleges cr further 
schooling opportunities. 

To investigate occupational plans^ students were asked to list 
three choices of occupations or jobs they wuld be interested in after 
finishing all schooling. In each school and grade^ two^thirds to three- 
fourths of the students listed three ctoices^ ap4 almost all others 
listed one or tt^. A larger percentage of junibrs than seniors listed 
three choices ^except in School A^ where they were equal)* This dif- 
ference between grades II and 12 was statistically significant at the 
#05 level of confidence. One possible explanation Is that 12th graders^ 

'ror details^ see Appendix^ Table Al^ the chapter on Evaluation Methods^ 
III, page 22, Of the 1889 students tested^ 85% had had no CAG experi- 
ence. The 15% who had already used CAG in February or March 1974 were 
almost all juniors^ the largest proportion of whom were in Schools D and 
E« 

^or details^ see AppendlKj Table A4i 

These high proportions of students expecting to go to college occurred 
under the CUNY policy of admitting every New York City high school 
graduate without charge for tuition or other entrance requirements. 
With the imposition of tuition^ there may be reductions in the propor- 
tions^ or changes in the colleges to which they eventually apply. 
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who are eloser to the necessity for decisioni about the future^ are less 
likely to entertain unlimited^ perhaps unrealistic^ Qptlons^ An alter- 
native but related possibility is that in the added year of maturing^ 
the aeniors have already narrowed their choices and now begin to focus 
on reallEable interests* Since slightly more Juniors than seniors also 
said they planned 00 post-high school education^ the "closer^to-decision- 
time" explanation aasumes more serious contemplation. 

Next we examined the first-choice occupations listed in response to 
the question above^ in terms of groupings^ and of specificity* There was 
a total of 341 different firsti^choice occupations; the most frequently 
listed groups of occupations^ in descending rank order, werei teacher^ 
secretary^ medical doctor^, nurse, lawyer, accountant, engineer, psycho- 
logist, journalist, business management, computer programmer, i^slcian, 
electrician, social worker, policeman, physical therapist, and dentist* 
Note that most of these occupations require post-high school training. 
The associated frequencies ranged from 181 for teacher down to 18 for 
dentist; there were 167 different occupations, about half the total, 
listed by single individuals. 

The nimber of responses and their frequency do not convey the 
variety of occupations listed — from medical Illustrator to horse trainer; 
nor the range in the degree to which students can be specif ic--from 
cytotechnologist or RCA repairman to "hospital work" to "painter." To 
quantify specificity of choice, we rated the occupations as very specific 
(i*e., listed in the Dictionary of Occupational Titles [ D,0,T. ]), medium 
(e,g*, teacher), or general (e.g,, work with chiidren) * The largest 
proportion, 64%, were very specific; 13% were medium, and 23% were 
general. These findings have Implications for the kinds of occupational 
information the students need; over one- third of then might usf help 
with narrowing and clarifying their choices, apart from others who might 
benefit from an opportunity to consider alternatives to their first 
choice. 

To detemine what factors were most Important in shaping students- 
decisions about their future college and occupational plans, respondents 
indicated, for each item on a list of 13 possible sources of influence 
(people, books, etc.), whether its inf luenae Ws "very much," "some," or 
"little or none." In all schools and grades, the students' "own in- 
terests" were consistently rated the highest as influencing decisions 
about the future, and their "own abilities" scored just slightly lower. 
Also ranking highly were the influence of "parents or guardians," and 
"someone who works or studies in (the student's) field of interest." 
Receiving intermediate ratings, in decreasing rank order, werei books, 
magazine articles, or stories; friends; teachers; and guidance counselors 
or advisors. The least influential factors (continuing in decreasing 
order) weres relatives, college catalogs, and TV or movtecn Although 
students rated piid ace counselors or advisors below the middle in rank 
order as sources of Influence on their decisions, results to be presented 
in Chapter vm on Impact of CAG on Student Users show that in January- 



Similar titles were grouped; e.g., "teacher" included special education 
teacher, music teacher, math teacher, etc. 
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February 1975 they railed counselors highest as a source of infomation 
about both colleges and careers. 

Given this picture of the students' college and occupational goals^ 
the next general area of investigation concerned students' perceptions 
of the college and career information they had received--or remembered 
receivlng--since the beginning of the 1973-74 school year. 

Almost all students said they had met individually ^ith a school 
guidance counselor Qr_ grade advisor at least once since Septmber 1973; 
these percentages were slightly higher for seniors than for juniors. 
Host of the help they received was ^Ith planning their high school pro- 
gran^ but it also inclined sane help with plans for post-high school 
education- The help from the guidance counselor or grade advisor seldom 
Involved occupational or Job plans; those reporting such help in each 
school and grade ranged from 3% to 19%. 

Many fewer students reported having seen a college advisor; about 
two-thirds of the seniors and a third of the Juniors said they had done 
so. As to the kind of help they said a college advisor gave th©n^ about 
half the seniors and a fourth of the Juniors said they received help 
with plans for further education* In both these respects^ the School E 
juniors were an exception^ equalling the seniors * 

The responses concerning college advisors as a separate group are 
somewhat suspect; from general observation^ students do not differentiate 
clearly ^ng guidance counselors^ grade or progrm advisors^ and college 
advisors^ even though in the high schools these people differ In their 
functions and their qualifications. The remaining questions^ discussed 
below^ pertained to advisors generally^ and did not require students to 
differentiate mong thOT. Students rated their opinion of the help they 
had received from the school advisory personnel^ and whether they would 
like to talk further with an advisor if tl^e pemltted. 

On a 3-polnt scale^ students Indicated how helpful the school ad- 
visors had been to them In planning what they would do after high school. 
Both juniors and seniors rated their advisors as Just below the middle 
of the scale In helpfulness. These findings may reflect the very great 
student load carried by most guidance and advisory personnel in the high 
schools^ and do not suggest that students thought poorly of the advisors. 

Indeed, when asked whether they would like more time to talk with 
an advisor^ over half the seniors and three-fourths of the Juniors did 
want more time to talk. If possible. Students who responded affirmative- 
ly could check more than one area in which they wanted help. More 
students wanted this entra time to talk about college, rather than 
occupational, plans; for both topics, more juniors than seniors said they 
would like to talk with an advisor. In general, the desire for more help 
was greatest at School A--the largest of the five schopls, and one with 
the relatively fewest niamber of advisors. 

These results suggest that students perceive more need for help with 
educational than with occupational planning^ and that Juniors express a 
greater need for help than seniors who were graduating in three months. By 
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the timB this data was collected^ the seniors had probably already 
obtained whatever help the school's advisors could give thmi^ especially 
with reepect to college; and their college applications were probably 
prooeiied by the high achoola* Up to March of the school year^ the 
advliors evidently give a greater priority to seniors^ which seems 
reasonable^ since there won't be another year available in which to help 
them* 

In stsnmary, most of the students in the pilot high schools expect 
to continue their education after graduation^ and can state possible 
occupational goala^ although with varying degrees of specificity* They 
do thirds about their future education and occupation^ guldeA:^rimarily 
by their own Interests and abilities. Most of thra had seen an advisor 
at least once for an Individual consultation and had received some help^ 
especially concerning plans for college. The results suggest a redun- 
dancy in the availability of educational Infomationj and at the same 
time a considerable need for occupational or career information. Of 
course^ educational planning necessarily inTOlves aspects of career 
planning^ although these students apparently perceive their most Im- 
mediate need as one for college information* The apparent redundancy 
in providing educational^ particularly college^ information may be a 
legitimate response to the students' requests. The results nevertheless 
point clearly to a need^ whether Identified by the students or not, for 
more planning information^ especially concerning occupations. Sizable 
proportions of students would like motm help to plan for their futures. 
They thOTselves emphasise help with college plans^ but exmination of 
their occupational choices suggests that they could alao use help with 
careei; planning. Regardless of what they need most^ In the eyes of 
many students a need for additional aid or information exists, whether 
CAG could help fill this gap was the major concern of this project. 



Main Features of the Guidance Information SystCTn 

This section briefly describes TSC's stated purposes^ the four 
flles^ the operation of the systm^ and some important terminology. 
Later sections will contain fuller details. 

The Guidance Information Systra was designed for use by high school 
students nationwide* TSC states that the Guidance Infomatlon System*, 

...makes it possible for students to explore 
large data files stored In a computer^ and to examine 
the ways in which their personal criteria for selecting 
colleges and occupations [affect] the range of oppor- 
tunities available to thra*... [The user] can. -sin- 
teract directly with the Information..*. [He] can 
change his mlnd^ and his instructions, at any point. 
This places the decision-making where It belongs, with 
the student. . . • 
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GIS consists of four separate data files of Infomation retrievable 
from the computer* 

1, The four-year college file (Col 4) contains information 
on approKlmately 1^600 colleges nationally, 

2* The two-year collage file (Col 2)^ which is similar In 

structure to the four-year college file^ contains infor*^ 

matlon On approxtoately 1^000 Junior and conmunlty 
colleges nationally, 

3# The occupational file (Occu) contains Information on 
about 1^300 occupations which were selected from the 
more than 20^000 in the Dictionary of Occupational Titles 
( D^0,T. ),1 

4. A scholarship and financial aid flle^ listing about 250 

scholarships. Since It Is a prototype and limited In size^ 
it was used less than 1% of the tlme^ so there will be 
few references to It In this report. 

The user sits at a temlnal^ which is connected by telephone line to 
a central computer. 2 Using the Instructions and code numbers provldt^., 
(s)he chooses one of the four files and types consnands on the terminal 
keyboard^ requesting information from the computer storage. The computer 
searches for and selects the requested information^ and relays it almost 
Instantaneously by typing its responses* The Instructions and the 
responses simulate a rudimentary conversation. 

The Guidance Information System has bean characterised as a direct 
interactive systOT^ without a monitoring capability. "Directly inter^ 
active" means that the user requests and receives information on the spot; 
thus he can see the results of his choices^ compare or change them^ and 
see the consequences of each additional instruction. "Without a monitoring 
capability" means that the computer keeps no permanent record of the 
interaction^ and the user's n^e (or other Identification) Is not stored 
in the computer* 

For this project^ each school had a "heavy duty" teletypewriter 



when an occupational description is printed out^ it lists titles of some 
"related jobs." GIS contains about 3^000 of these related occupational 
titles with D.O.T, niaabers. Since these "related jobs" are generally not 
in the data flie^ the student can consult the D.O.T, for a description of 
ma«y m&^e occupations , IRDOE provided the schools with D.O.T , 's, 
2 

TSC'a computers are in Hanover^ New Hampshire* a foreign exchange line 
was brought to New York City so that^ for our schools^ the link-up was the 
equivalent of making a local telephone call. The CAG project had one such 
telephone link-up^ the use of which was rotated ffltong the schools on 
different days. From September 1974 on^ ITOOE conducted a similar project 
in CUNY cOTanunlty colleges using GIS and more telephone link-ups^ thereby 
increasing the total amount of potential access to the computer. 
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terminal — the work-horse model—which provided a hard copy of the Inter- 
action in printed form. This printout is the only pemanent record of 
the use. Carbon paper was used thrQUgfiout the project; the schools 
usually gave the original copy to the student and kept the carbon copy 
On file* All students who ware at the terminal wanted to keep a copy* 

The "Student Study Guide" constitutes the main tool for using the 
systott. This booklet contains explanations of the instruction letters 
which tell the computer to perfom its functions^ directions for using 
the various files^ and a list of all college and occupational charact- 
teristlas users need to obtain information of interest. Although the 
Guide is reusable^ several hundred were purchased for each of the high 
schools* There Is also a "User Instruction Manual^" containing Infor- 
mation for counselors and alphabetical lists of colleges and occupations^ 
with their GIS code nLmabers (and^ for occupations^ the D.Q,T* nraiber) * 
Students would use this Manual if they wanted the code number to obtain 
a description of a specific preselected Job or college In GIS^ or if they 
wanted to know whether an occupation or college Is included In the data 
files , 

GIS uses a narrowing logic- as more specifications are considered 
either by inclusion or cKclusion^ the original pool (of^ say^ 1^300 
occupations) becomes smaller and smaller* The college and occupational 
files are similar in structure. (There are also Important differences^ 
which will be discussed more fully in later sections.) The stored 
information is organised by Jharacterlstics ^-qualities or features which 
describe an occupation or a college. Characteristics define what types 
of information can be retrieved. All characteristics are coded by 
number. The characteristics are grouped into broad categories . "All 
women" is an exmple of a characteristic in the "Coeducation" category. 
The college files contain more characteristics (and categories) than 
the bccupational file^ and thus provide more choice. 

To use GIS^ the computer unist be Instructed in what to do with the 
characteristics; these instructions (or coimnands) are signified by a 
letter. Thus^ the computer can be instructed either to consider only 
"all women" colleges^ or not to consider colleges with an all-fCTiale 
student body. To operate the system^ the user must know at least 5 of 
the 12 commands or instructions. These five basic comaands Instruct 
the computer to: 

1. Consider or Include (colleges or occupations having) 
this characteristic. 

2. Remove colleges or occupations having this character- 
istic. (Do not consider those with this characteristic.) 

3. Delete or erase from consideration ii characteristic 
previously specified (l.e.^ "I've changed my mind"). 

4. Print the names (if there are 25 or less) of the colleges 
or occupations that meet the prior specifications* 

5. Print detailed Information about a specific college or 
occupation (identified by its GIS code number). 
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Using these commands^ alone and in combination^ the files can be 
used in two ways — to make searches or to provide dBScriptions , In a 
search^ the user selects characteristic's that are important to him and 
derives a list of jobs or colleges that meet the requirements he has 
specified. For a description^ the user requests details about a par- 
ticular college or occupation by uaing the fifth conmiand^ Print. Either 
or both of these types of uses may occur in one computer interaction. 
Searches could occur first^ to narrow the range of choices^ and could 
be follQwed by a description of one or more of the choices, A search 
requires a clearer understanding of the operating instructions (ter- 
minal commands) and of how the systan is conitmctedp Flares I and 2 
show^ respectively^ armotated sraple printouts obtained from an occu- 
pational search^ and from a description of an occupation. 

One of the more significant differences between the college and 
the occupational files is the distinction between selectors and 
descriptors . In the college file^ selectors and descriptors are the 
same; that is^ any characteristic can be used as a specification cri- 
terion for selection as well as appearing in the description. This Is 
not true of the occupational flle^ where some characteristics are only 
for description and cannot be used as Input* For exmple^ consider the 
description of "jobs that are mostly performed outside"; the user cannot 
ask for (select) a listing of occupations that are "mostly outside." 
Such Information is^ however^ available in a description of a specific 
occupation* This difference between selectors and descriptors In the 
occupational file makes It more difficult to grasp the way the occupa- 
tional file works^ as compared with the college flles^ which are simpler, 

Another factor affecting the CAG experience Is whether the student 
is or Is not present during the computer Interaction* This Is not a 
feature of the system as much as a function of how it was used in the 
schools. This report refers to direct use^ meaning that the student was 
present during the computer Interaction^ even though a staff member may 
have operated the terminal. Indirect or batched use means that the 
student had previuusly submitted in written form, which a school staff 
member processed in his absence. Generally^ batching was done(on an 
on-line day) before or ifter regular school hours^ or at any time no 
students were scheduled Batching was sometimes done In Instances where 
students failed to keep their scheduled appolntaents The advantages 
and disadvantages of both types of use will be discussed In detail In 
Chapter VI, pp* 58 ff. 



One other command also produces a description^ but one that Is much 
more detailed. For any college (or occupation)^ this conmiand produces 
a list of all characteristics (in code nmbers or In English) true of 
the given college"-i*e.^ those characteristics which the college has* 
In the college files in particular^ such a list is lengthy; If done in 
English^ it is very tlme'-cons™lng^ and if done In numbers^ the user 
must decode them, it is also possible to obtain a partial list. 
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FIGURE 1 

SMPLE INTERACTION EOTIjOYING THREE BASIC GIS COMMANDS 
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FIGURE 2 

DESCRIPTIVE MODE: SMPIE OF AN ACTUAL CIS PESCRIPTION OP AN OCCUPATION 
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EVAUJATION METHODS 



The evaluation activities were designed to accomplish two general 
purposes i 

1. To systematically monitor the project operation^ so 
as to identify areas of potential successes and prob*' 
lems that facilitate or interfere with the Implanen- 
tation of a computer-related guidance activity In the 
public high schools; and 

2# To assess the effects of such a syatOT on student 
declslonHnaklng and on staff and student reactions. 

To these ends^ much effort went into planning an evaluation that 
would provide evidence on which to base future decisions and the re^ 
direction of maladaptive practices* We wished also to use techniques 
and Instruments that required little staff time and that might prove 
helpful to the school staff. Infonnal feedback suggested that certain 
of the evaluative procedures did meet the schools' own administrative 
recordkeeping needs. Considering the mount of information we ra- 
quested of the llaisonsj they spent very little of their CAG time 
(reported mean^ 6%) In gatharlng and maintaining evaluation data. 

Figure 3 shows the instrmnents used for assessing system operation^ 
use of the system^ and the quality of the data files; also Includad are 
the dates of administration or use^ and eKplanatory comments* Fl^re 4 
shows^ similarly^ instruments used to obtain participants* reactions 
(upper half) and to assess students (lower half)* Much of the informa- 
tion in these two figures will not be repeated in this text* Tables 
Al and A2 in the AppendiK give fuller details about the students who 
were assessed* 

IRDOE staff made 10 introductory and btaff orientation visits 
(2 per school) prior to February 1974. During the three project periods 
there was a total of 97 man-days of site visits; the majority of these^ 
especially in Periods 1 and 2^ were on Dn^line days to see what was 
occurring and how the GAG operation might be improved* Off-line day 
visits were usually for data collection and informal discussions. We 
often helped students and staff at the tet^ inal^ and learned a great 
deal during these visits* The liaisons rt rted that these sessions 
were valuable to than; they frequently req.asted our presence when 
they expected visitors^ anticipated problems^ or wanted to try some- 
thing new* 
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FIGURE 3 
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Observations 

(in man-days; by IRDOE 
staff) 



Copy of Printout (borrowed 
from liaisons) AW 

User Sign-In (kept by 
terminal operator) 



User Sign-In AND 
TSC AutOTated Susmarles 
(without student nOTes) 



Date(a) 



Pds.* 
12 3 



Feb* 1974 



3/74-6/74 
(Pd. 1) 



Comient 



Visits on both on-line and off -line days. 
Pd. 1^ N-45| Pd. 2, N-27; Pd. 3^ N^25 
(Including visits for student testing). 



For each use. Printouts plus User Stgn-In 
gave name^ grade^ file^ date^ search/descri| 
tlon^ and direct/indirect data* Required 
matching printouts with User Sign-In to 
obtain total users and uses. 



Obtained times^ plus Information as above^ 
except search/description^ without having 
to reconcile the Infomatlon with copies 
of printouts (although IRDOE continued 
this process for two months as a check) . 



TSC Automated Summaries 
(with student nanes) 



Pds, 



2, 3 



TSC modified its sunnnary program so as to 
pemit typing user name into terminal. 
Gave all time and user information^ and 
all in one record. Usefulness depends on 
the fidelity with which identification is 
typed in at the time of use. 



Hardware Monitoring 
Checklist 

(kept by liaisons and 
IRDOE staff) 



Pds. 

1 2 3 



For each on-line day. Described type and 
extent of malfunctions and estimated re- 
sultant time lost. 



Data Files Output 

(Analysis by IRDOE staff; 
also^ liaisons' reactions) 



Feb. 1974; 
(May 1974 

and 
April 1975) 



Exmination of printouts and comparison 
with information from other sources gave 
recency and accuracy of infomatlon In 
the flles^ and characteristics used in 
searches. (Also based on ratings from 
Liaison ^estlonnaires^ and Information 
Rating Cards. See Fl^re 4.) 



^Period 1 = February - June 1974 . 
Period 2 ^ September-December 1974. 
Period 3 - January - June 1975. 
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FIGURE 4 

INSTWJMENTS FOR OBTAINING PARTICIPANTS* REACTIONS 
AND FOR ASSESSIffiNT OF STUDENTS 



Instrument or Technique 


Date(s) 


Comment 


Participants ' Reactions 






Information Rating Card 
(kept by liaisons^ 
IRDOE ataff^ and any 
other user) 


Pds^ 
1, 2, 3 


For noting any unusual (good or bad) 
outcomes or experiences^ by file and 
characteristic; e.g*^ output omissions or 
logical problems with system construction. 


Log Day 

(kept by IRDOE staff^ 
and at School D by 

T H 3 "* ^ #^ ir^ 1 

ixaxson ) 


May 1974 
(1 entire 
on-line day 
per school) 


For each student user that day. Gave a 
description of a typical day. Served as a 
basic source of data on student reactions. 


lilalson Questionnaire 


5/74 
4/75* 


Mailed* all 8 liaisons responded # 
Mailed; all 5 liaisons responded. 


Principal's Questionnaire 


5/74 
2/75* 


Mailed; all 5 principals responded* 
Mailed; all 5 principals responded. 


School Librarian 
Ques tionnalre^ 


4/75 


Mailed 6 (2 to School E) ; all responded. 


School Personnel 
Questionnaire* 


4/75 


Mailed to 18 guidance counselors and 
teachers involved with CAG; 13 responded 
(with at least 1 from each school). 


Student Assessment 






Initial Survey of Student 
Activities and Plans 
(given by classroom 
teachers^ liaisons^ 
and IRDOE staff) 


March 1974 


Time required^ 40 mln, 1889 Ss were tested 
in English classes^ or In economic classes 
in School D, Gave initial picture of 
student views^ although a few (mainly in 
grade 11) had already used CAG. Analysis 
is based on grades 11 and 12. For details^ 
see AppendiK^ Table Al, 


College Office Applications 
Records* 

(analysis by IKDOE staff) 


6/74 and 9/74; 
6/75 


For all Identifiable college applicants or 
applications in the 5 schools. Processed 
5366 applicationi in 1974^ and 4899 in 1975 


Tests of Impact of CAG^ 
Vocabulary; Decision-* 
Making (plus Preferred 
Sources of College and 
Career Information) ; 
Self College^ Ideal 
College^ Self Occupa- 
tion and Ideal Occupa-^ 
tlon Questionnaires 


Jan# /Feb, 
1975. 
Given by 
IRDOE staff; 
some liaisons 
assisted. 


Time required^ less than 1 class period. 
Liaisons were asked to select English 
classes equally divided between users 
and non-users; School A tested Ss in 
homerooms. In Schools D^ and Ss 
tested took either Vocabulary or 
Decision-Making^ plus 1 other test. 
School C classes took Vocabulary and 
Decision-Making^ but no other test. 
For details^ see Appendix^ Table A2y 



*Datfl not previously reported. 
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The basic information on student use of the computer system was 
.o]btained in different ways at different times. In the first monthj 
to gut basic data on which files were used and the mode of use^ we had 
to borrow the carbon copies of the printouts from the liaisons. The 
carbons were linportant to the evaluators as well as to the schools^ 
which used them in various ways--primarily as a basis for followup 
discussions with students. All schools hoarded the copies^ and were 
reluctant to lend th&m to the evaluators* In several instances where 
the printouts were not available for data analysiSj this was usually 
because they had been given to counselors. When we obtained the most 
complete set of carbon copies available, we matched than with the 
User Sign-In records, which (if completed) gave the date, users' nraes^ 
grade level, and the type of GAG experience (direct or indirect mode) . 
Even when the schools very carefully maintained both sets of records, 
the matching process invariably showed that several students were not 
accounted for on one or the other record* It becOTie clear that a 
better method was needed for identifying users and for sunsnarizing 
data on use. 

In March 1974, through a contract modification^ TSC began provid- 
ing daily automated sunmiarieSj giving an account of all requests made 
of the computer^ with tljpfles, for each school; but the sunsnaries con- 
tained no student names. These accounts provided useful Infomatlon 
about time, and furnished some check on data obtained from hand anal- 
ysis of User Sign-In records; however, identification of individual 
students, and whether the use was batched or direct still had to come 
from matching the automated summaries with the User Sign-In records. 
While an Improvement, this still did not resolve the problem of uniden- 
tifiable uses* In addition, there were omissions in the automated 
summaries that usually occurred as a result of the breakdown of the 
central computer, or the shift to another computer that was not pro- 
granaed for automated smranaries. 

Beginning in September 1974 (Period 2), we requested a change in 
TSC^s smmary program to permit student nanes and other descriptive 
information to be typed at the terminal at the time of use. The infor- 
mation appeared on TSC's suimaryj although it did not go into the 
computer; this change facilitated the task of data collection and 
analysis* In those instances when not all of the identification Infor- 
mation requested was entered at the tenninal^ it would not appear on 
the TSC line-by-line simnttiary. Generally, however, both time and user 
information became available on the ssme record, and matching of two 
separate data sources was no longer necessary. 

Because all the basic inforTOatton about system use was not com- 
plete, especially in Period 1, our analyses are perforce only for those 
Students and variables for which we could determine the necessary iden- 
tification* It seens ironic that in a computer project so much of the 
data collection and analysis had to be done by hand. Manual record- 
keeping and data handling are time -consuming and less than fully 
accurate* We know, for example^ that our counts of multiple users of 
GAG are not precise because of duplications and misspellings in student 
names* With no citywide student identification numbering systan, 
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completely accurate counts are not possible^ despite whether the anal- 
ysis is automated or done manually. 

The Hardware Monitoring Checklist provided a record of equipment 
malfunctions^ the frequency and duration of breakdowns^ and responses 
to calls for service. Most liaisons completed these records fairly 
carefully in Periods 1 and 2 when every malfunction was perceived as 
major^ but much less so in Period 3^ by which time the liaisons had 
become quite competent about dealing with equipnent-related problems* 
As these lessened over tima^ the liaisons raised more questions about 
implanentatlon and the underlying theory and construction of the data 
files (a*g,^ how they were compiled^ how colleges and occupations were 
classified^ atc#). 

Such questions were discussed during observation visits^ as well 
as at two group meetings of the liaisons held in April and Septanber 
1974* At the first meeting the liaisons described how they oriented 
and scheduled students for CAG use and how they followed them up after 
use. This discussion led us to develop the one-page Log Day form^ as 
a way of describing a typical on-line dayp IRDOE randomly selected 
one day per school in May 1974^ on which all CAG activities^ Including 
students^ reactions on the spot^ were systematically noted* For each 
user that day^ CAG staff^ assisted by an evaluator-recordar^ summarized 
what occurred during the use^ as well as preliminary and followup 
activities p Log Day information was a basic source for description 
of student reactions* 

The quality of the data files was assessed from our own indepen-- 
dent checking^ from ratings of the categories and characteristics by 
liaisons^ and from notes on the Information Rating Card* On this card, 
any user (but mainly staff) reported unusual outcomes^ noting the file 
and characteristic used* 

To document changes in career and college counseling practices 
that are attributable to CAG is a very complex task^ because of the 
variety of people in different capacities who perfom such functions 
in the high schools. Instead of attempting to assess the effects of 
GAG on time allocated to these guidance functions^ we collected 
opinions^ ratings^ and Judgments by means of questionnaires mailed to 
school staff members. In May 1974, on a lengthy questionnaire, the 
liaisons rated^ described^ and evaluated the project, including staff- 
ing^ administrative support, training, materials, quality of the data 
files, and effects of CAG on students. They also indicated how much 
time they allocated to CAG, and how the CAG time was spent. In April 
1975 they completed a similar questionnaire. The principals, in a 
brief questionnaire each spring, expressed their opinions and described 
their reactions to the project* Since staff members other than those 
assigned to the CAG unit took part in project activities, we asked 
liaisons for names of such people, and sent out a general School 
Personnel Questionnaire* Also near the end of Period 3, the librarians 
received a questionnaire specific to thOT. The response rate to all 
questionnaires was 100%, except for the School Personnel Questionnaire, 
where about three-fourths of the 18 surveyed responded. 
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The Initial Survey was given in mid-March 1974 to all students 
in classes selected by liaisons as being representative of the entire 
population In grades 11 and 12 (see Appendix, Table Al) , Regular 
classroom teachers, plus liaisons, administered most of the Surveys * 
The 1063 juniors and 826 seniors tested constitute approximately a 
25% SOTiple of all studencs in the respective grades* Very few of the 
seniors tested had had CAG experience (overall, 3%); about a fourth 
of the Juniors had used CAG (range, from 13% In School B to 40% in 
School D) * Where Ns pemitted, we compared users and nonusers, but 
found few differences at this very early stage. The Survey results 
describe the Initial status of the students to be served by CAG, in 
terms of their attitudes and future plans. (See Chapter II, pp* 8 ff.) 

To obtain objective evidence of the effects of CAG on student 
behavior, we first considered that college applications might reveal 
changes attributable to CAG* But because of the difficulties with 
respect to individual schools' recordkeeping practices, we could not 
compare CAG users and nonusers directly, and instead resorted to a 
comparison of the 1974 and 1975 graduating classes. The supporting 
reasons will be eKplalned in the section on Analysis of College Appli- 
cations* (See Chapter VII*) It is clear, however, that there are 
many ambiguities In the college applications data. 

In reviewing the very positive project results obtained In Period 1, 
the Board of Education and First National City Bank requeated additional 
data on the effects of CAG on students, so as to be able to reach a 
tmre Informed decision concerning its future use* To g^lde the direc- 
tion of this undertaking, IRDOE and a representative from the Division 
of lligh Schools Jointly proposed Impact criteria* In relation to the 
criteria adopted, we devised direct cognitive and attltudlnal measures 
of thu impact of CAG on students. The purpose of these tests was to 
assess effects of CAG use on decisionmaking^ vocabulary, student atti- 
tudes, and students' knofwledge of^ and description and organisation of 
Information about colleges and careers* 

Early in 1975 (Period 3), imOE, assisted by liaisons, gave these 
tests to users ^nd non^users In the same classes. No class took more 
than two of the tests. (Table A2 in the Appendix gives details.) For 
the testing, we had requested that the liaisons select classes about 
equally divided between users and non-users overall, 53% (389) of the 
740 different students tested had used CAG (range, from 36% at School C 
to 62% at School E) . These students were all in grade 12, except for 
School E, where 84% of these tested were in grflde 11. Of the 389 
different users, 52% had had only indirect (I.e., batched) CAG experi- 
ence, and 57% had not used CAG since Period 1--6 to 12 TOnths prior to 
the testing. It is possible that the non-users had some eKposure to 
CAG in group orientation sessions, but none had used the teminal, 
either directly or Indirectly. 

The same statistical design was used for analyzing results of 
both the Vocabulary and Decision-Making tests. On both tests the 
score was the number of correct responses. To deal with the question 
of possible initial differences in ability level between users and 
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non-'asers, we had hoped to use student honors status as a control vari- 
able. (It was not possible to ask liaisons to use honors status as a 
selection variable^ In addition to asking for equal division between 
users and ncn-users within classes.) As it turned out^ no honors stu-. 
dents were tested at Schools B and and at the other schools^ the 
cell Ns were very disproportionate and some were very small. To 
statistically compensate, we used a multiple regression program 
("Glypoth'*) to analyze significance of differences,- 

For the final analyses^ we combined honors users and non-honors 
uaers and compared them with honors plus nonhonors iion-users ^without 
dlitlnguishlng between honors and nonhonors students- As explained 
in Appendix we are willing to assume that no important initial 
ability differences existed between users and non-users that would 
account for the obtained results. 



We realize, of course, that many other questions can be raised 
about the initial cpmparabillty of users and non^ueers we will dis- 
cuss some of the possible variables in connection with the test 
results , 



The remaining impact tests dealt with student attitudes toward 
the computer^ and other sources of college and career information, and, 
by means of four questionnaires, with the way students described and 
organized Information about colleges and careers. There were four 
open ended-questionnaires^-Self Collegej Ideal College, Self Occupa^ 
tion, and Ideal Occupation* On the ''Ideal** questionnaires, students 
were asked what questions an ideal counselor should ask of students 
looking for a college or occupation; the "Self" Instruments asked 
students where they might consider going to college or what occupation 
they may pursue, and asked them what were the important features of 
their choice. These Instruments had no "correct" answers; their reli- 
ability and validity had not^been established; and they were given 
only in Schools B, and E, For these reasons we did not analyze 
the results statistically; nevertheless, the findings are clear enough 
to stand by themselves. 



The Glypoth program handles the unequal cell Ns by using cell means 
as the population estimates^ and yields F ratios which are interpreted 
In the sane way as those obtained from analysis of variance. For a 
description of the Glypoth analysis^ see* Gross, A*L*, Costa, N,D., 
^ Steckler, J*F* A Fortran Prograjn for Hypothesis Testing in the 
General Linear Model, Journal of Education and Psychological 
Measurment, 1974, 34, 133-135. 

2 . 
The attitude questions^ which were placed at the end of the Decision* 

Making test, asked the student to select, from each of the two lists 

of sources presented^ the three sources (s)he considered most valuable 

for obtaining college and career information. Respondents were Included 

in the analysis only if they gave three responses to the question- For 

each question, proportions ware obtained, by school, of the number of 

users and non-users selecting each listed source as one of the three 

most valuable* 

3 

The Self College Questionnaire was given only in School D; thereafter, 
we gave the ideal College Questionnaire, since it proved to be easier 
to score and interpret. ^ 
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Each of the four open-ended questionnaires provided space for 
writing 10 responses. The questionnaires were scored for the total 
number of undupltcated statements and the total nmnber of different 
categories employed. The number of statOTents^ or amount of infoCTna- 
tion^ represents the student's ability to describe what he knows about 
colleges or careers. The nimber of categories or the variety present 
in the statements^ reflects how the student organizes the information* 

Responses to each questionnaire were analyzed Independently by at 
least two scorers^ without knowledge of whether the respondent was a 
user or non-user, A statement was defined as containing one piece of 
information. Two statements by the sa^e student containing the same 
information counted as one statement (e.g.^ "The school is coed^" 
and "It has male and fOTale students.") Categories were not imposed 
on the data^ but were generated by the actual content of the responses. 
Specific categories were narrowly rather than broadly defined. We 
used GIB classifications and labels whenever possible^ which permitted 
analysis into GlS-related and non-GIS-related categories. For eKample^ 
the two statements^ "The college requires high marks for admission^" 
and "The college requires high BAT scores" were placed in the one CIS 
category of "competitiveness."! (TSC groups these two characteristics 
under this classification.) Any one student might make two or more 
statements within the sme category; thus the number of statanents 
per student was usually larger than the number of categories (s)he 
employed . 

For the two occupation instruments^ another breakdown was Into 
Job-oriented as compared with personal-oriented categories. (The two 
college Instruments did not generate responses divisible Into this type 
of impersonal vs. personal grouping*) Job-oriented responses were 
worded in terms of Job or work situation characteristics (e.g.^ "This 
occupation requires arithmetic ability")* Personal-oriented responses 
were phrased In terms of the student' s background or abilities (eeg., 
"You nMSt be able to work with numbers") # or in terms of the character- 
istics of the hypothetical student counselee named In the question 
(e,g,^ "Are you good In math?"). Some responses were not scorable as 
job- or personal-oriented^ either because they were "neutral" (e.g,^ 
"Interview people in the field" or "Go to the library") or Irrelevant 
(e.g.^ "Work is hard") or ambiguous (e.g.^ "Where do you want to work?"). 

Although decisions are difficult concerning what constitutes an 
undupllcated statanent and what kinds of statements constitute a cate- 
gory^ we made every attenpt to remain consistent and fair. Analysis 
of any open-ended questionnaire presents similar difficulties^ but 
this is the price to be paid In obtaining responses which are not 
stimulus -bound . 



Note however^ that these two responses (if made by a single student) 
would be counted as two different statanents since they do not cover 
Identical information — the former exmple refers to high school grades^ 
while the latter refers to scores on a specific test. 
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CHAPTER IV 



PROJECT imLEt^NTATlON 



The aim of this chapter Is to provide some flavor of CAG in action 
in the five demonstration high schools through a description of how the 
project was staffed^ how the time was allocated and used^ and how each 
school operationally responded to the demands and challenges provoked by 
the computerized project* 



Proiect Staffing and Administrative Support 

From its inception^ the CAG project received a warm welcome. Prin- 
cipals of the high schools eagerly awaited its installation^ were 
cQDperative in finalising arrangements^ and implemented the project in 
thalr schools In a manner that facilitated effective demonstration. 
What helped Inestimably^ in our Judgment^ was that throughout the three 
time periods (February 1974 - June 1975), the Division of High Schools 
allotted one teaching unit to each of the pilot schools.! (A teaching 
unit represents the equivalent of the average salary for one regular 
classroom teacher* A licensed counselor, for exmple, represents 1.3 
teaching units^ while a paraprofesslonal is the equivalent of .7 of a 
teaching unit.) The schools used their teaching allotment in different 
ways, but each principal appointed at least one liaison, who was re- 
sponsible for coordinating the CAG activities, and who served as IRDOE's 
primary contact. Most principals selected regular teachers for this 
purpose^ releasing them from their other duties* Since all teachers 
must do some classroom instruction, these liaisons could not be assigned 
to CAG full time; some portion of the allotment thus becfflne available 
for other assistance* 

With the eKception of School B (at which a guidance Intern was 
hired as a full-time CAG liaison), all other schools assigned CAG re- 
sponsibility to relatively young^ regular teachers, in addition to 
having an average of about seven years' teaching experience In the 
school system, they also had special experience as college advisors, 
career counselors, and/or program advisors. All had demonstrated com- 
petence and concern about quality education. According to their own 
estimate^ the liaisons were selected because they had a Job history of 
personal involvement with students, a high degree of organizational 
skills^ and a close relationship with the schools' college and career 
advlsfflaent services . 

In our judgment, it is rare to meet a group of people so interested 
and involved in a project, and so willing to spend time in self- and 

^As previously noted, for the period from September 1975 - June 1976, 
this allotment was halved and each school received the equivalent of 
half a teaching unit. « 
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program«appraisal. Informal discuaslons with the principals suggested 
that^ as they said^ the liaisons were indeed ''some of our best people," 
With the prevailing scarcity of dollars for education^ the principals' 
choica of such competent personnel provides one Index of their conmitment 
to the CAG project* 

The individual at each school who was primarily responsible for 
liaiaon duties did not change *1 Some of the other changes in. assignments 
from period 1 to Periods 2 and 3 reflected reallocations of projact 
duties to better suit individual school needs; many reflected cltywide 
budgetary curtai In Perlqdi 2 and 3 there reduced nimiber of 

professional staff invoived with CAG at Schools and A (as well 

as a clear reduction in the OTMJunt of ttoe all 5 sctools spent at the 
terminal-^see pp. 30 ff*)^ although this was not reflected in the 
liaisons' reported astimates of the proportion of time they devoted to 
all aspects of CAG. Their estimates for Period 1 ranged from 16% for 
the School C primary liaison to atoost lOn for School B; in Period 2 
their estimates ranged from 17% for School C to 87% at School B* 

The staffing patterns at the Individual schools follows 

School A . In Period 1, liaison responsibility was shared by an 
English teacher with grade advising duties (the primary liaison) and a 
health education teacher* Several hours of paraprofessional tlme^ and 
a few student monitors (who received "service credit") completed the 
staffing. For Periods 2 and 3^ the grade advisor and a full-time para- 
professional conducted the GAG operation; student monitors continued^ but 
to a much lesser degree. The terminal was placed In a small private room 
in the college office* 

School B . In a?.l 3 project periods^ a guidance intern was hired for 
the CAG project* Since this school had the most elaborate and varied 
career education progrms^ he was responsible for coordinating GAG with 
all other career and decision-making activities. In Periods 2 and 3^ in 
which the terminal was moved from the Career Education office which the 
intern shared with two other members of the staff to an office of his 
own^ he had the clerical assistance of student monitors* 

School C * CAG responsibility In Period 1 was shared by a college 
advisor-English teacher (the primary liaison) and a licensed counselor 
for career guidance. In Periods 2 and 3 only the college advisor was 
actively responsible; since the college office Itself was more minimally 
staffed during this time^ the liaison had to assume additional responsi- 
bility for its functioning^ reducing the actual time devoted to CAG, The 
location of the terminal in the college office facilitated her duties^ 
but contributed to the career counselor's problems in coordinating her 
career office advising duties and CAG during Period 1. 



For 1975-76j all schools retained this same parson with the exception 
of School B which transferred responsibility to a teacher of business 
education* 
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School D . In Period 1^ CAG activities were shared by a collage 
advisor-English teacher (the primarily active liaison)^ an occupational 
counselor^ and tvo half- time paraprof essionals who manned the project in 
the early evening hours. The paraprofesslonals were recent graduates of 
SahooX enrolled in a CUNY college^ and intereated in computer tech- 
nology. For Periods 2 and 3^ the profeaslonal level responsibility was 
entirely that of the college advisor^ who similarly to School C devoted 
incraasing mounts of time to the college office; the time of the para- 
professlonals was also reduced to about one- third of the Period 1 amount* 
Since the terminal rmalned in a location most accessible to the college 
.off Ice^ ^ iQcceast-ng .usjq. was. made .by the other-, college -advisors * _ . . 

School E # In Period 1^ a math teacher who was responsible for 
School E's College and Career Office was selected as primary liaison^ 
assisted by a counselor responsible for a special career project which 
ended in June 1974, They had the assistance of a part-^tlme secretary as 
well as a paraprofesslonal two days a week* Later^ all professional 
responsibility fell to the primary liaison^ but the secretarial and para- 
professlonal assistance remained at the sane level. The terminal was 
located in the librarian's of flce--»also shared by the paraprof essional- = 
several floors away from the College and Career Office^ but adjacent to 
the career section of the library. 



In most schools CAG was established as an adjunct to the college 
office^ primarily because career advisement services were^ in general^ 
not well developed and the college office was one of the locations (and 
services) that was accessible to and familiar with large mnounts of 
student traffic. In all schools CAG functioned independently although 
cooperatively with the other related departments and staff* The liaisons 
were responsible to the principal and/or to an assistant principal for 
guidance. By the end of Period 1^ principals were wong CAG*s warmest 
supporters p They wanted to "keep the progrmi golng^" and they wanted 
access to the computer more than the scheduled two days a week. The 
principal of School in an unsolicited letter to the project director^ 
expressed it as follows- "The CAG project has been of greats almost 
inestimable value to us. It has added a dimension of knowledge and... of 
motivation. It has stimulated our entire. . .operation. There has been an 
almost galvanizing effect. The progrm should be expanded to other high 
schools if funding can be found. Certainly^ it must be kept here." At 
the end of the project^ the principals still rCTiained strong supporters^ 
eKcept for the principal at School C. Although this principal indicated 
that [his] interest In CAG semad to diminish over tlme^ he also stated 
that^ In his observation^ student users did benaflt. 

The liaisons were unanimous in their hlgh^ positive rating of the 
attitudes of their school administrators in Period I; all said the ad- 
B^nlstrators were very much interested In the project and had observed 
its operation on both on-line and off-line days. Only two principals 
(Schools B and D) requested occasional forroal reports from the CAG liaison 
(number of students served, percentage using each of the four files^ etc.)^ 
but discussion and personal observation showed that all principals were 
aware of the problms and progress of the project. Their "very much 
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iiitereatmd" attitudes^ according to the liaisons' reports^ were apparent 
in the amount of time and effort the adralnletratlon aKpended--lntroducing 
CAG to the paranta and confflaunlty^ encouraging dOTonstratlons for (and use 
by) people In other schools and agencies^ writing articles for the school 
newspaper^ and rootlvatlng Involvonent of other school staffs Of great 
toportance to the liaisons was the fact that the principals established 
few constraints^ facilitated liaisons' requests for supplies^ and granted 
permisalon to mploy class time for student orientations^ terminal use^ 
and testingi By the end of the project^ liaisons at Schools and D 

viewed the administrators as staying wore in the background* The liaisons' 
TOpit. frequent c administrators relied on CAG to fulfill a 

valuable function* 



Orientation, Tr a ining, and Allocation of Staff Time 

Training sessions were conducted in each school near the end of 
January 1974^ during Regents week* One or two representatives from TSC 
and iroOE staff conducted the trainings The sessions ran from three to 
as many as six houra^ with one to twenty people present* Attenders 
usually included the principal^ the assistant principal in charge of 
guidance, tha llalson(s)^ counselor(s)^ advlsor(s)^ staff of other special 
pro jecf f 5 oaiiiide interested group repreaentativea^ and faculty from other 
depar^^f s^i'S (e*g»^ math^ data processing^ etc;), 

ttm intent of the sessions was to ffflniliarize the school staff with 
^Am capabilities of the system^ to orient than to the conceptual basis 
of Ui€ progran^ and to teach them basic use of the equipment (how to make 
corit'^nt with the computer)^ the functions of the terminal commands ^ and so 
on* The Student Study Guide was the basic training tool* After group 
discussion and an on-line dMonatration^ each participant who wished to do 
so could try out the systOT individually* 

With rrme on-line practice it takes less than an hour to learn the 
essentials f:*perating the systra; acquiring full aKpertlse and finesse^ 
however^ tak^f: a little more time. At the end of the initial training 
session^ most u.articlpants at the schools felt they had a general under- 
standing of *;h'i system and expressed confidence about the hardware. They 
recognized thar. it would be difficult to learn to operate the system from 
the printed maiierials alone^ since much of the necessary information was 
not clearly our lined In either the Student Study Guide or User Instruction 
Manual. As In the case of School however^ reading the materials In 
advance clearly facilitated the initial sassions* 

At these meetings^ or at ones scheduled inmediately following^ we 
also worked with the liaisons^ explaining necessary details^ including 
explaining the scheduling of terminal time^ priorities for student use^ 
whv;*: to do in case o£ difficulties^ and the recordkeeping procedures to be 
maiii -.alned* We also continued with on-line practice^ covering certain 
rftf<n©sents^ such as how to correct a typing error^ shortcuts in searching^ 
J^ad other ways to maximize obtaining Information quickly and efficiently. 
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Further Informal training (and mutual learning) continued with each 
visit lOTOE staff made to the schools. Initial analyses of use of the 
various cocmands suggested that maxima 1 efficiency and complete under ^ 
standing of the narrowing aspects of the GIS program were still not 
universal in February 1974; however^ by the middle of Period 1^ all such 
problems were overcome* The high school liaisons demonstrated competence 
and masteryj frequently raising questions about the sj/stKn that we were 
able to answer only with the help TSC provided* 

The liaisons' responsibilities were many and complex. They included 
et^e:^if>g to -t>he €AS p^'iorities^-s^tivati^ reor^iting students^ 
orienting or eKplaining the systra's capabilities to tharij helping operate 
the terminal or arranging schedules for terminal operators^ following up 
students^ maintaining records and forms^ and (especially in School A) 
conducting deTOns trations for outside groups* In Period 1 the liaisons 
spent the largest proportion of their CAG time (l*e.^ the time they devoted 
to all aspects of the project) in orientation activities--recruiting and 
explaining in classrooms^ with groups^ and with individual students. Next 
most time was spent in followup activities after students had received the 
printout. Estimates of the anount of time liaisons spent personally 
operating the terminal ranged from 10% (Schools A and D) to 30% (School B) * 
The most significant change In Period 2 was In School where the liaison 
estimated that she devoted 50% of her ttae to operating the terminals- 
reflecting the reduced availability of the paraprofesslonals , In general 
in Period 2^ proportionally less time was spent in orientations (which 
include recruiting)^ with more time spent in followup activities with 
students* 

At each school^ other staff becfflne involved in the CAG operation . 
We observed school staff manbers learning to use the system on their own^ 
and helping to retrieve information for students (Schools A^ and E) . 

They also helped with orientations^ interviewed students^ and referred 
them to the liaison* They aided in followup activities^ explaining the 
output and helping students locate additional services for more information. 
Many staff members used the teminal to request information about colleges 
and careers pertaining to their own subject matter area--as a learning ex- 
perience for themselves and to meet needs of students who might say^ for 
example^ "I like math; what can I do with it?" These others also used the 
terminal to obtain information as parents^ and for friends who were parents 
of school^aged children. According to the principals^ staff members of the 
schools were impressed and enthusiastic when they noted increased student 
xootivation* Again^ School C's liaison and principal were the only ones to 
note lack of general interest fflnong the school staff during the 1974-75 
school year* Even the few staff who felt ''the information wasn't terribly 
accurate" continued their involvement and showed their interest in the CAG 
concept by continuing to use It. 
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SGhedullng and Use of Terminal Time 

Although the teminalg were Installed by January 4, 1974" and the 
Schoole had received their first supply of Student Study Guides, the 
phones were not put in until the last two weeks of the month* 2 During 
this time students were taking Regents exaninations, and the new tern 
was scheduled to begin in February, Thus^ the first few project days at 
the end of January were used for practice. Staff of the schools and any 
students available were encouraged to explore and experiment with the 
Guidance Information System* These days were hectic because the schools 
were using the system contention; that is, a school dialing in^ 
computer would get a busy signal if another school was already on-line^ 
and this happened frequently* The contention experience was very brief ^ 
but it was obvious that advance scheduling of on-line days was necessary. 

For Period 1 we prepared a calendar assigning each school exclusive 
use of the system one day a week; the specific weekday was rotated monthly 
so as to equalize any disadvantages associated with particular weekdays. 
While this procedure worked adequately^ the schools did much trading of 
days or half -days* The liaisons IndtcatBd a preference for having one 
regular weekday assigned to them^ so that they could r ?sr^ their time 
longer in advance and students could become more awar^^ r:f their day" on 
the computer (and thus would be less likely to forget ; si-^^ments that 
had been scheduled In advance) , There was no problem iii leslgnlng a 
set-day calendar for Periods 2 and 3; according to the liaisons' reports^ 
this arrangement was very successful^ and they did much less trading of 
days than in Period 1* Modifications in scheduled days were maintained 
centrally^ every request being cleared throu^ IRDOE* 

Days and Mean Hours Used * The data to be presented next pertains to 
the scheduling and use of on-line days and the time used to process stu- 
dent requests for Information* (It Includes only student use of the 
terminal*) The total number of days scheduled and used^ as well as total 
terminal tlme^ are not directly comparable across the three periods^ 
because of the differences In duration of the periods. The two measures 
that can be compared over time periods are the days used expressed as a 
percentage of days scheduled and mean hours on-line per day. 

In Period 1^ all weekdays were prescheduled^ and the only "extra" 
time was Saturdays and Sundays, In Periods 2 and 3 extra weekday time 



TSC rented terminals from Data Access Systems^ Inc. (DASI) * This led 
to one school's nickname of "Daisy" for its terminal; other schools also 
adopted nicknames and we had "Herble" at Herbert H, Leimian High School, 
as well as "Alphonse", The particular model (KSR-33) we used types at a 
speed of 100 words per minute^ but is very sturdy^ and records the Inter- 
action as printed copy* In operation^ however, it is noisy, 
2 

A general freeze on phones for all New York City agencies necessitated 
special permission for the Installation of the five private phones* When 
not in use for CAG, the phones were locked. At the and of June 1974 and 
1975^ phone service was temporarily suspended and terminals were stored. 
All equipment had to be reactivated each September (1974 and 1975). 
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on-line was often available to the high schools on request. These days 
cmm from a companion CAG project operated by IRDOE in four CUNY community 
colleges*^ Some schools (A and D) requested extra tljme^ while others 
(Schools E and G) rarely did. In siaimary^ the five high schools requested 
and used 13 extra days or parts of days in Period 2 and 24 eKtra days or 
parts in Period 3^ the Spring smester. The schools sometimes used the 
extra time instead of^ and not in addition to^ their scheduled days. They 
raquestad extra time not necessarily because they did not have enough^ but 
rather because they did not have the "right'* time (l-e.^ days when staff 
and students were available). Most often they requested and used half-days 
or less--for example, when the staff did not have a full day to devote to 
CAG. This fact suggests that, at least in some schools, a schedule of two 
half-days per week might maxtoi^e terminal use to a greater extent than 
the scheduling of one full day. 

A small portion of the on-line time scheduled in advance was not ^ 
uaed*-*10% days in Period 1, 5 days in Period 2^ and 21 days in Period 3* 
The reasons for not using scheduled time were similar to those for re- 
questing eKtra days--nonavallablllty of CAG staff or of students (during 
graduation rehearsals. Regents or final eKaminatlons, etc.). In Periods 2 
and 3, some unused days also resulted from shifts in schedule that meant 
departing from the re^larly assigned weekday ;3 the school would sometimes 
simply forget to get on-line on the rescheduled day. 

Table 1 (p. 32) shows the on-line days used as a percentage of the 
days scheduled in advance. In Period 1, School E stands out because it 
used every scheduled day; the other schools used 84% to 89% of their 
scheduled time* In Periods 2 and 3, with the extra days available, most 
schools used more days than had been scheduled. School G again differs 
from the others* it used 86% of its scheduled time in Period 2 and 73% in 
Period 3--*much less than the other schools* 

In terms of mear ^^ours per day spent on the terminal (Table 1) this 
time was by far the gioatest in Period 1, and dropped to half that amount 
or less in Periods 2 and 3# The only eKception to this general trend was 
School 1, which maintained almost the sane average in all three periods 
(5%, 5, and 5% hours) . School C had the lowest average per day throughout, 
with a very sharp drop in the 1974-^75 school year. 

The raasons for the decreases in mean hours per day in successive peri- 
ods seffli attributable to particular situations In the schools. In School C, 
and to soma extant D, the Increased responsibility of the CAG liaison 
for othar school act ivities was a factor. The School A liaison offered as 

^Stii^lated by the early positive feedback from the high schools^ the New 
York State Education Deparment's Bureau of Two-Year College Prograns and 
the Division of Occupational Education Supervision funded a demonstration 
of CAG at the conimunity college level (VEA Grants Nos* 75-2-587, 76-2-421). 
The projact was cDutlnued in 1975-76, expanded in 1976-77 (VEA NO. 77-2-381). 
2 

Sehoola C and D accounted for 14 of the 21 unused scheduled days in 
Period 3, 
3 

Shifts in schedule cmie about as a result of equalising days for those 
schools whose regularly scheduled day fell on a holiday. 
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TABLE 1 

rarailNAL TIME SCHEDUmD AND USED FOR STUDENTS, BY SCHOOL AND BY 
TDffi PERIOD (TBffi IN HOURS, TO NEAREST QUARTER -HOUR)* 



Perloa 


School 


All 


A 
A 






D 


£ 


Schools 


Period 1 (5 months) 














Days Seheduled 


1 Q 




1 Q 




1q 


93 


Days Used 


17 


16 


15% 


16 


18 


82% 


Days Used as % of 
Days Scheduled 












89% 


Total Tijae in Hours 


99% 


98 


78i 


103% 


98 


477% 


Mean Hours per Day 


5% 




5 


6% 


5% 


5% 


Period 2 (4 BOnths) 














Days Scheduled 








13% 


12 


67 


Days Used** 


14% 


14 


12% 


13 


14 


68 


Days Used as % of 
Days Scheduled 










117% 


101% 


Total Time in Hours 


35% 




15 






205% 


Mean Hours per Day 


2% 






3 

— — ■ 


5 


3 


Period 3 (6 months) 














Days Scheduled 








22 


23 


113 


Days Used** 


31 


22 


16 


25 


23 


117 


Days Used as % of 
Days Scheduled 


135/s 


96% 


734 


114% 


100% 


104% 


Total Time in Hours 


79% 


45% 


14% 


70% 


119% 


330 


Mean Hours per Day 


2% 


2 


1 


2% 


5% 


2% 


Total, All Periods 














Days Scheduled 


55 


56 


54% 


54% 


53 


273 


Days Used** 


62% 


52 


44 


54 


55 


267% 


Days Used as % of 
Days Scheduled 


114% 


93% 


81% 


9n 


104% 


98% 


Total Time in Hours 


214% 


187% 


107% 


Ilk 


289 


1013 


Mean Hours per Day 


3% 


3% 


2% 


4 


5% 


3% 



*A11 times in this report should be considered as underestimates, 
especially in Period 1* 

**Includes SKtra daysi see eKplanatlon in text. 
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aa explanation that by Period 2 the student's GAG on-line experience was 
usually only a part of a longer Individual coniultation^ which CAG helped 
to initiate, Thui, lesa time was spent at the terminal and Mre with 
atudenta. In Schools B and E the fact that the staffing rmnained relative- 
ly eonstant throughout the project may help to account for School E's 
maintenance of a high level of terminal use^ but it does not explain 
School B*B decline in tours used per day* The School B decrease in 
Period 2 was^ however^ not as great^ relative to Period 1^ as the decreases 
in Schools A^ and D. And in Period 3^ School B nearly doubled the 
ftmDunt of indirect use (processing requests in the absence of the students)^ 
a proGedure which requires leas time. 

Although the schools used fewer hours per day in Periods 2 and 3 as 
compared to Period 1^ they tended to use a greater proportion of their 
achaduled days^*101% and 104%^ respectively for periods 2 and 3 as compared 
to 89% in Period 1* Thus, to sot© extent^ the fewer nimber of hours used 
per day may be explained by the fact that the schools took advantage of 
the availability of extra days by using less time on more days. (This also 
explains the proportions in excess of 100% noted above.) This lends 
support to the suggestion that for some schools^ scheduling more half -days 
may be better than whole days* Other conditions^ to be discussed in sub- 
sequent sections^ also contributed to the changed patterns of use. 

Not reported in Table 1 was the time used for purposes other than 
handling student requests. At least a total of 12 hours in Periods 1 and 
2^ and more than 14 hours in Period 3^ was used for staff eKploration 
and demonstrations for classroom teachers^ student groups from other 
schools^ parents^ and a large nimiber of agencies and individuals from the 
educational and business ^industrial coniminlty. 

E qu^iETOen t _Ma 1 fun^ * All computer-'related projects suffer from 
hardware malfunctions and related peripheral problans* Our original In- 
structions to the liaisons were that they should call TSC at the first 
sign of a problem^ and could expect a response from them within 10 minutes. 
The possible malfunctions included problCTs with telephone llnes^ tele^ 
typewriters^ and the central cOTaputer. The occurrence of a problem did 
not mean that the schools experienced ^or reported) lost time. As an 
eKample, when they checked "garbage" as a problem^ this might not have 
Interfered with the operation^ as long as the essential parts of the 
printout were readable* The Hardware Monitoring Checklist^ to be com- 
pleted for each scheduled on-'line day or half*day that was used^ gave 
information about the nature of the problem^ the degree of disruption it 
caused^ and an estimate of the timm it took to be corrected « 

Different malfunctions caused the schools different problmns. Some 
telephone-related problems^ for exmplej had to be referred to the phone 
company^ while others could be dealt with by TSC« Similarly^ some 
terminal-related problems were corrected by a call to TSC while for others 
(such as those resulting from blown fuses)^ TSC had to call in DASI. In 
this latter type of problem^ the rCTiainder of the on-line day might be 
lost to the schools since DASI repairmen could not service the teminal 
until the following day. If a school lost the use of an on-line day^ It 
usually could not process any requests until Its next scheduled day the 
following week (unless the achool could use extra days In Periods 2 and 3)* 
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Since we esttaiated the mount of time they probably would have used CAG 
that day^ it is somewhat unfair to aKmlna ''Mean Reported Hours Lost per 
School per Month" in Table 2^ but these means do at least give a relative 
picture of the mount of disruption caused by equipment problems* On the 
average^ each school lost about two to three hours in each of the start-up^ 
months^ but only a half-hour by the final month (June 1974) of the period. 

The main point illustrated by the data in Table 2 (p* 35) is that 
reports of malfunctions and time lost because of thra were greatest in the 
start-up months-^nanejy^ February-March^ and September 1974--and diminished 
considerably thereafter. Liaisons submitted 82% of the raquasted Checklists 
In Period 1^ and 717o in Period 2. The number of days with malfunctions 
reported dropped frOT 55 in Period 1 to 21 in Period 2^ or from two- thirds 
to about one-third of the days on which reports were requested. By 
December 1974 we were receiving almost no Checklists. For this reason^ 
malfunction analysis was not possible (nor necessary) in Period 3; by then, 
these problems were minor* Our observations and discussions with liaisons 
indicated that on days for which they submitted no Checklists, very little 
time was lost and few or no malfunctions occurred* 

During the first few project months, an equipment breakdown of any 
kind caused a flurry of activity, including calls to IRDOE and calls to 
TSC (as instructed). In February and the first part of March 1974, any 
and all problems were usually first addressed to IKDOE; nearly always, we 
either called TSC or DASI or askedthe school to call TSC directly* By 
mid -March 1974 the schools were making their own direct calls; only in rare 
instances when a new type of problem occurred, or one requiring a change 
in scheduling, did they notify us in advance* After the first few months, 
the schools no longer reported short intervals (5 to 10 minutes) of time 
lost, taking such brief Interruptions much more in stride. By May 1974 
staff at some schools became so blase about malfunctioning equipment that 
they did not even call TSC, because they had learned certain techniques for 
handling many problms thanselves. 

That the perception of problras changed more drastically than did the 
occurrence of problems is supported by a comparison of the liaisons' regular 
dally Checklist recording of problems with their ratings of probl^s and 
service in their questionnaire at the end of Period 1. The frequency of 
occurrence of problems noted in the Checklists ranked, from most to least, 
as follows: telephone-related, garbage, cutoffs, and password rejections. 
The end-of-*perlod questionnaire ratings cited garbage as the most trouble- 
some overall* This result suggests that the principle of recency was 
operative, since garbage was the most i^ecent, as well as the most frequent, 
problra Just prior to the ratings* 

According to the Checklists, the schools made 21 calls to TSC for 
assistance between March and June 1974.2 For a third of these calls, the 
schools reported that TSC either did not respond at all (N-5) or did not 

^The reported total of 63% hours lost for periods 1 plus 2 is a minimum 
estimate, since it does not include short intervals of unknown frequency ^ 
and duration. 

^The Checklist was not available for the entire month of February and, 
therefore, no accurate count is aval lab let 
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TABLE 2 

REPORTED Tim LOST DUE TO EQUIPMENT MALFUNCTIONS IN PERIODS 1 AND 2, 
BY MONTH, ALL FIVE SCHOOLS COmiNED* 







Reported; 




Period 


N Scheduled 
Days Used 


N 

Days 


Days with 
Malfunc- 
tions 


Total 
Hours 

Lost 


Mean Hours Lost 
par School par 
Month 


Pariod 1 (5 montha) 












February 


16% 


18 


18 


13% 


21 


March 


20 


18 


15 


16 


3% 


April 


15 


13 


9 


1% 


% 


May 


22 


14 


8 


3% 


% 


June 


9 


5 


5 


2k 


% 


Total 


82% 


68 


55 


36% 


- — _ = ,_ _ 














September 


11 


9 


6 


12% 


2% 


October 


21 


12 


6 


6% 


1% 


Novoflber 


14% 


12 


7 


5% 


1% 


Decamber 


13 


9 


2 


3 


% 


Total 


59% 


42 


21 






Total, Periods 1&2 


142 


110 


76 


63% 





*A11 time calculationa are to the nearest quarter-hour. Malfunctions 
did not necessarily involve lost time; e*g.^ in April 1974 malfunctions 
were reported on 9 days^ but the total tijne reported lost was only 1% 
hours for all 5 schoolsp 

Reports on the Hardware Monitoring Checklist were requested only 
for scheduled on-line days^ so this table does not include any extra 
days used in Period 2 (see text). Some schools^ by trading^ used the 
terminal for half-days^ for each of which a Checklist was requested. 
Thus^ "Reported! N Days" can exceed "Numbar of Scheduled Days Used," 
Of the total of 142 scheduled days, 140 were whole days and 4 were 
half-days^ for a total of 144 days on which malfunctions could have been 
reported. For Period 3^ there were too few reported problems to warrant 
analysis of malfunction data. 
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reapoad within one hour (N-2) . Service improved In Period 2^ as indicate 
by TSC'i returning all 113 reported calls. At the. end of Period 1^ the 
liaiions rated TSC's service-"promptness of response^ courtasy^ and -^ob- 
ligingness*" These ratings tended to be good or excellent* In the end-of- 
Period 3 ratings^ however^ two Itaisona rated TSC as poor In respect to 
forewarning about difficulties or interruptions of service, ^ 

Liaisons' attitudes toward TSC^ Data Access^ and IRDOE suggest that 
some of the success of GAG Is directly attributable to the Interest and 
attention the liaisons felt they received* Coronents like the following 
provide confirmation for this vlewi "Sometimes the problem seemed insur- 
TOUntable until TSC gave me sound^ easy-to-follow advice that solved 
everything"; "Any questions or problCTis that arose were handled expedi- 
tiously by those people who coordinated the project"; and "I found the 
IMOl staff extraordinarily helpful and supportive," 



GAG Operation in the Pilot Schools 

This section will describe the varlad ways In which the schools 
implemented the GAG projactp While all faced stollar overall restraints 
such as staffing^ one day par week access to the computer^ and priorities 
for student use^ each school responded in different and somewhat unique 
ways. As already explained^ each principal exercised his freedom to 
divide the allocated teaching unit into actual personnel as he chose and 
in response to schoolwide concerns. The scheduling of on-line days^ like- 
wise^ was adapted by the liaisons who wanted Increased flexibility to meet 
their school needs. Originally^ IKDOE set diffarent student use prloritie 
for Periods 1 and 2. In Period 1 we asked that GAG be used primarily by 
juniors seeking occupational information^ while in Period 2 the priority 
was to be for seniors selecting colleges*- However^ we Instructed the 
liaisons to disregard these priorities if they Interfered with optimal use 
of the system. As will be seen In Table 9 (page 67), overall^ the schools 
tended to focus on Juniors in the Spring semesters (Periods 1 and 3) and 
on seniors in the Fall. 

Gertain other factors^ some alluded to earlier^ also affected the 
strategies developed by the individual schools* Although each had to 
develop procedurea to deal with such comnon concerns as the nature of the 
Student Study Gulde^ for exOTple^ as well as recruiting^ orienting^ and 
following up students^ there were other more unique considerations* These 
included school organisational factors (such as size of student enrollment 
and length of the school day)^ location of the teletypewriter terminalj 
and the schools' educational cllmate. 



^or exraple^ TSC periodically took the computer out of service for over- 
hml or maintenance checks^ but did not (or In some cases^ could not) 
notify IMOE or the schools in advance as to when this would happen* 
2 

In period 3 there were no priorities; th^^ schools could make their own 
choicas* 
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Since the Guidance Information System is designed for national use 
by high school students^ the presumption was that the students ^ with 
the help of an adult^ could read the Study Guide^ make choices among char- 
acteristics^ learn the necessary machine commands^ process their own 
requests^ or prepare a summary sheet which they could use at the terminal 
or leave for batching (1.6.^ processing in their absence). 

At the early meGtings with the liaisons^ there was a good deal of 
speculation about the suitability of the Study Guide for the New York City 
high school student population represented by the five schools. Concern 
was expressed that the language level was too difficulty the format was too 
complex^ many of the characteristici were not relevant to the students' 
needsy and the instructions and conmands were confusing. Some of these 
problema^ however^ involve the way GIS is constructed' others appeared more 
menable to solution. Possible solutions for these difficulties were dis- 
cusaed^-including the provision of an index^ rewriting especially confusing 
pasiageSy or complete revision. The original contract^ however^ ran only 
through Dec^ber 1974^ and since its purpose was one of assessing feasi- 
bility^ we did not undertake any major revisions. Instead^ we encouraged 
and helped each school to make those adaptations it found necessary. We 
also discussed some of these concerns with TSC^ who revised the Study 
Guide three times between late 1973 and June 1976.1 

As it turned out^ the liaisons- initial feelings about the difficulties 
of the materials for students proved to be correct. The schools coped with 
these difficulties In various ways^ all by attempting either some modifica- 
tion In the Study Guide and/or with student worksheets and/Dr by working with 
groups or individual students. 

From the project's Inception IMOE encouraged direct use of the com- 
puterized information retrieval systmi on the assumption that the advantages 
of having the student present outweighed any disadvantages. (We will 
return to a discussion of these points In Chapter VI ^ Student Use of the 
Computerized System^ and in Chapter VIII, Impact of CAG) * While agreeing 
that direct interaction might prove TOst beneficial^ the liaisons developed 
styles of operation more in keeping with such factors as the size of the 
student body and the number of free class periods for students. 

„ The primary advantage of indirect use is that it permits serving many 
more students per unit of time; its disadvantages are that the student 
cannot influence the process^ and special orientation and/or followup 
sessions are needed. These sessions could^ and Indeed nwst often dld^ 
take place on off-line days. In Period 1 all sclwols^ except School 
processed from 71% (School A) to 79% (School D) of the uses in the absence 



At the beginning of each project period^ IRDOE supplied new sets of Study 
Guides to the schools. Although revised^ on inspection the revisions 
appeared minor and the inadequacies in fomat and content that characterlEed 
the earliest editions persisted. It was not until the 1976-77 year that' 
substantial modifications were made- this edition represents a great many 
improvements^ including a Glossary of Tems^ a simplified and more logical 
fotinat with larger type and fewer columns per page^ and clearer^ more 
complete definitions of characteristics. 
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of the itudent. Not only li this a reflection of the schools' desire to 
provide sarvice to as many students as possible, but it also reflects other 
organisational considerations. If a sctool wanted to encourage direct use 
by stuuenti^ this usually entailed scheduling appointments in advance for 
the on-line day* Not all students have free periods^ but those that did 
could use the terminal then, if the room was open and supervision was 
available. In general, most schools found little demand for direct stu« 
dent use after school hours; the fact Is that many students work after 
school. Getting students in and out of clasi to go to the terminal 
involved mich paperwork (e.g., ^writing passes) for the CAS staff; moreover, 
at first the liaisons were reluctant to ask that a senior be excused from 
an English class to use the teminal. As ttae went on, however, and the 
value of the systOT becane apparent^ the liaison would schedule on-line 
appointoents during class time. In those instances where students forgot 
appolnttaents, their requests might be batched and the indirect users asked 
to return to pick up their printout and discuss it* 



Reflecting differences in stmcture, organisation, and emphasis, the 
Individual schools evolved their own strategies for recruiting and 
orienting students, handling student use of GIS^ and followup after ob- 
taining printouts/ Next we will describe seme of the unique aspects of 
Implamsntatlon in each school^ and the ways in which they adapted the 
general procedures to meet their particular goals. 

School A . As already noted, this school had a very meager career 
education program, and because of the length of the school day it did not 
offer mich in the way of after-school activities. At the beginning this 
school was the one most concerned about having the student process his 
own request but also moat concerned with loss of class time. The GAG 
s taff worked late on every on-^llne day, but in this, the largest of the 
schools, students had no or few free periods and a high percentage worked 
after school. Thus, a large ai»unt of batching was done. 

In Period 1 School A expertaented with several procedures. To re- 
cruit students in February and March 1974^ the liaisons went into the 
College Bound classes and distributed Study Guides, simplified smnmary 
worksheets^ and specially prepared sets of revised machine conmiands to 
interest students in "Project Decision." In the classrooms, the liaison(s) 
reviewed the Study Guide and helped individuals prepare the sunraary sheets 
which were to be brought to the terminal at the appointed time. By mid- 
• March, the liaisons felt confident enough to attCTipt to enlist the interest 
of the cooperative work-study students (generally, those of lesser academic 
ability) ^-at first individually^ but later in small groups. The liaisons 
indicated that small group orientation was preferable because of the peer 
support. For all three periods, the lirison(s) did all orientation; in 
Periods 2 and 3 the liaison was assisted by other professional staff 
members* 

In Period 2, to implement the priority for seniors to use the college 
filea^ the rCTiaining liaison (a grade advisor) worked closely with the 
college advisor; the location of the terminal in the college office facil- 
itated the flow of such referrals. In concert, the liaison and the college 
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advisor decided to discourage use of CAG by the large proportion of 
students interested only in CUNY^ since more reliable information about 
CIMf is eailly available In the college office. Instead^ they encouraged 
eKploration by students interested in non-CUNY schools^ with the result 
that there were fewer users than in Period 1 (and considerably less 
batching). The liaison called in each student whose class standing made 
competitive collagea (i,e* ,non-CUNY) a possibility* Once their requests 
were processed^ the system was opened to all other students. 

By Period 3 few^ if any^ classroom orientations were conducted. Word 
had spread through the school and sufficient numbers of students came on 
their own to use the terminal. This freed the lialsouj who established a 
"mini^career resource library" near the terminal^ starting with the D,0 ,T# 
IBDOE provided to each high sctool; the library was stocked with all the 
materials she could locate free of charge* The fact that word -^of -mouth 
advertising caused School A students to initiate asking for an appointment 
was "very unusual in this school," "Most students^" according to the 
liaison^ "do not actively seek advice" l-*although by 1975 they "were 
stopping 'the computer lady' in the hall and asking me questions*" 

School B . Of the flve^ School B had the major career emphasis-- 
including a good career library and participation in several other career 
orientation programo supported by outside agencies | In addition^ over 
half the school population was in the Career Academy, Because this was a 
newly-built school it did not have a full graduating class In June 1974. 
Thus^ CAG was used mora by students in lower grades than the other schools. 

In February 1974^ the ^idance Intern initiated CAG by providing two 
full days of orientation to large groups of 10th and 11th graders. For 
these^ and all other orientations^ he xeroxed the Study Guide^ cutting 
and pasting pages to reduce foraat problfflas. The initial recruitment 
effort^ similarly to that of every other sclwol^ was to recruit students 
who had t^en PSATs or who expressed interest in going to college. This 
liaison^ like the others^ focused on higher ability students initially 
because the liaisons were not COTpletely comfortable with GIS^ and the 
college data files were easier to manipulate. 

Group sessions dealt with general issues (e.g.^ what are liberal arts 
colleges)^ as well as with students' individual problCTis, The liaison 
spent a good deal of time with each student at the terminal^ and had many 
direct users operate the terminal themselves. In Period 1^ School B's 
liaison had the largest proportion of direct users* According to the 
liaison^ he was much less successful in interesting non^college bound 11th 
graders. With each student^ he spent a great deal of terminal time in 
eKploring and discussing college and career alternatives. Students who 
could not have direct Interaction were Invited to submit a summary work- 
sheet; after batching^ the liaison ataost always arranged a followup 
interview with students. In the Initial project months^ nwst batching 
resulted from students' failure to keep their appolnbnents . 



According to the results of the Initial Survey (March 1974) ^ however^ 
School A's students had the greatest desire for more help. 
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The llalaon also eatablished tha CAG experience as part of a new 
Dacleion*Maklng course offered by the guidance departaent. There were 
efforts to coordinate CAG activities cloaely with the ichool's career 
progr^s. By the end of March 1974^ he had begun a COTmunlty program 
drive Inviting School B parenti and students to come in together in the 
evenings or on Saturdays^ to use GAG. 

In Peri^ 2 the terminal was mved to a private location that served 
as the liaison's office which becme a "center'- for college and career 
advising* During every visit we observed many students In the office^ 
preparing sumary worksheets and looking through brochurei and catalogs. 

The one notable change in procedures In Period 3^ which affected 
outcofflesj was a radical shift to batching « According to the liaison^ "In 
order to serve more students we have almost eKCluslvely indirect use of 
CAG*" 

School G * Perhaps relying on Its reputation as one of the more 
"acad^ic" public high schools^ School C attaipted no revision in the 
Study Guide or auxilllary student materials* Inlttally (in February and 
March 1974)^ recruitment was directed to the higher ability students; 
the college advisor-liaison conducted orientations In only two classes^ 
after which "word of CAG spread^ and self-referred students crae to the 
college office asking to use the cOTputer*" They were encouraged to 
borrow Study Guides^ or read thm there^ and to complete a smnmary work-^ 
sheet if they wished « Many students did not [have to] fill out a work- 
sheetj either working the temlnal themselves or asking one of the three 
advisors In the college office for assistance. School C students had free 
periods and often cone to the college office at the end of the school day. 

Toward the end of the first period^ CAG staff began to reach out to 
students of lesser academic ability^ those in remedial classes as well as 
those in the "modified" (easier) curricultmi* Most of these were recruited 
by the career counselor-liaison^ and most of their requests were batched* 
The occupational file was used mainly to obtain descriptions of specific 
occupations for use with a career education group or for publication In 
the school's newspaper. 

In Period 2^ as has been mentioned^ only the college advisor worked 
with CAG. She dld^ of course^ help students who were Interested in career 
Information^ but with the serious staff cutbacks In the college office 
there was an overall decrease in use. Most of the students who used GAG 
sought access on their own. To facilitate the CAG operation^ as well as 
the large volume-of-applications-to-non'*CUNY c^ asked 
each student who wanted to see a college advisor to use GIS first . 

According to the liaison^ use of CAG "fell off during Period 2 
because most of the potential college^bound 12th graders had already 
applied to college and had had CAG experience as juniors" in Period 1. 
School C personnel felt that^ for college information^ "the systCTi is 
moat valuable to students in the sprang semester of the Junior year." 
Whlle this may be one important reason for the great decline In use In 
Period 2^ usage in Period 3 was also low* Despite the on-paper allocation 
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of the teaching unit alloment to CAG^ it was quite clear from visiting 
the school that the liaison's first loyalty was to the college office; she 
had mich lass time to complete IRDOE's data collection forms and to re- 
cruit students and help process their requests; moreover^ she indicated 
that CAG might Increase the office's already heavy work load by leading to 
more college applications to be processed and more tests (e,g,^ SATs) to 
be arranged , 

School D . At this school the liaison(s) modified the Student Study 
Guide extensively^ more extensively than at any other school. The primary 
liaison produced a much-shortened guide to the college and occupational 
files^ by including only those categories she considered most important 
(e*g*^ tuition^ location) to students. She told the students^ however^ 
that the complete Study Guide was available^ and Invited them to review it. 

In Period Ij drientatlon took place in English classes. Both liaisons 
(the college advisor and the career counselor)^ and at least one parapro- 
fesslonal^ visited each class and discussed GIS^ giving a full period of 
orientation* Largely because of the time allotted and the fact that there 
were three (or four) staff people present to give students individual help^ 
all Interested students prepared sramiary sheets. The students wrote out 
their requests and Interests (in words ). The paraprofesslonal later 
coded these requests Into the appropriate comiands and characteristics. 
School D had requested^ paid for^ and received a somewhat more complex 
terminal model«-one with a papertape feed--that enabled pretaped requests 
to be processed automatically. As a result of this school's Interest in 
and familiarity with computers^ only about a fifth of the interactions in 
Period 1 were direct. Most direct uses^ and all batched ones^ were pro- 
cessed by the paraprofesaionals in the late afternoon and early evening 
hours . 

The primary liaison also operated the terminal, She^ moreover^ spent 
a great amount of time early in Period 1 exploring the construction of the 
syat^. Together with the paraprofesstonals^ they first brought to our 
attention some of GIS's inconsistencies and inadequacies. The liaison(s) 
screened all printouts before returning than to the students; if it 
appeared necessaryj^ she requested the student to come in and discuss it. 
Students also initiated appointments with the advisors and counselors 
after receiving their printout. 

In Periods 2 and 3 activity at School D was low; however^ the amount 
of direct use Increased greatly. Both occurred as a result of the same 
citywlde cutbacks in staffing. As In School GAG personnel on-paper 
assignments remained about the sane as In Period 1^ but they actually had 
much less time for CAG activities. The counselor-liaison was inactive and^ 
according to the college advisor-liaison^ potential college-bound seniors 
had previously explored the college data files* Many 8th term students 
enrolled in the mini-school began to use the occupational file, and almost 
all students were self -referred. 

School E . School E was the only one of the five that maintained a 
"College and Career Office.'* This office was staffed by a mathematics 
teacher who assumed CAG responsibility as well* With the help of part- 
time paraprofessionals and a secretary she tried many approaches for 
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devising a smooth flow from orienting to following up students* Because 
the school was so small (about 1/3 the size of School A)^ by the end of 
Period 1^ almost all seniors had had GAG experience. 

During Period 1^ the liaison invited all students seeking or re- 
ferred to the office for advising to review the Study Guide and complete 
a suninary sheet"color*coded by file. Both she and the paraprofessional 
went into classes to provide orientation^ and there were referrals from 
other staff ar4 counselors. The liaison estimated that approximately 
one-third of the students heard about the syitOT by word- of -mouthy another 
third signed up as a .result of classroom orientation^ and the rOTialnder 
represented referrals from faculty and from somiunity-wlde publicity. 

In PeriQd 1^ In the liaison's eagerness "to give a good thing to 
everyone^'! School E batched 3 out of every 4 requests; but if students 
came to the terminal during a free period on an on-line day^ the liaison 
encouraged than to use GIS i™aediately. Either she, but more likely the 
paraprofessional, operated the terminal. By Period 2, the liaison placed 
somewhat more emphasis on having the student present (i.e,, on direct use). 
^4e observed that it was not uncoraaon on on-line days to see an entire 
class lined up^ each student waiting his or her turn* 

As can be seen in the chapter on Student Use (Chapter Vi ), School E 
saw the most students, was most consistent in the aowunt of on-line time 
they spent each scheduled day, used all scheduled days and rarely requested 
more^ and were rigorous In completing data collection forms and maintaining 
their own CAG records. The only changes from Period 1 to Period 2 and 3 
concerned unproved (i.e., better systematized) procedures p 

If the student did not come In to pick up his batched printout. It 
was sent to his homeroom. (All 5 schools sent the printout to the home- 
room class In similar situations.) School E, however, sent the carbon 
copy to the student's guidance counselor In an attempt to involve the 
guidance department in the followup process. Another example of a follow- 
up procedure that was tried was having the liaison return to the class- 
room, distribute the class's batched printouts, discuss the output, and 
encourage another round of use. 



In summary^ In this chapter we have attCTpted to provide a descrip- 
tion of the various ways in which five pilot high schools implemented a 
computer-assisted Information retrieval systffln. Some of the need for 
adaptation was a direct result of such things as the way GIS is constructed 
and the way the Study Guide was written* Other variations resulted from 
the limitations imposed by the mount and duration of funding--lncluding 
one day per week access. Still others reflected unique and salient aspects 
of the pilot schools--thelr slEe, the range of administrative support, the 
ability level of the student population. Even the age of the physical 
plant had an impact^ affecting (at the simplest level) the location of the 
temlnal a^ its accessibility to staff and students. The variation 
enabled Important differences to become apparent and thus provided us a 
broader base on which to make suggestions, recomendations, and predictions. 
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CHAPTER V 



AMLYSIS OF Tm GIB DATA FIIJSS 



Before presenting the data describing the ninaber and nature of the 
studenti who used GIS--and Its Impact on thQn--lt may be well to examine 
the nature of the information a student obtains when using the system* 
Although the purpose of the CAG project was to assess the concept of 
computerizing college and career infomation for high school students 
and staff^ the test of such a concept necessarily involves demonstration 
with a specific system; the usefulness of any such systOT depends 
largely on the kind and quality of information contained in the files^ 
its accessibility^ and its meanlngfulness to the student In relation to 
his needs and Interests, One concern in this ehapteif is the adequacy 
of the infomation and of the system to the New York City high school 
student population. 

The analysis of GIS is based on examination of printouts^ and the 
Student Study Guide and User Instruction Manual; Independent checks on 
the accuracy^ recency^ and completeness of the information in the data 
files; observations; student reactions; and liaisons' perceptions and 
ratings. The discussion will necessarily include comments or implica= 
tlons about the Student Study Guide which contains all categories^ 
characteristics^ explanations^ and cocmands for using GIS. The consensus 
raong the liaisons was that a laifge proportion of the students could not 
use GIS without help, part of the difficulty was in the verbal presen- 
tation in the Study Guides compounded by the lack of explanation about 
how information in the data files Is classified and interrelated. 

The GIS files use f narrowing or "Inverse pyrmld'' fomat. The 
Study Guide^ In its general oveifall explanation of each of the files^ 
does not stress this narrowing or decision-making logic. With one major 
exception^ the college files and the occupational file are similar* In 
the construct and use of the college flles^ however^ no distinction Is 
made between characteristics that can be used as input and characteristics 
outputted* This is not true of the occupational file; In this file a 
distinction is made between characteristics that can be used as input in 
a search foif occupations (selectors) and characteristics that describe an 
occupation as output (descriptors). Thus^ in the occupational file^ the 
user can obtain more descriptive information about an occupation than 
(s)he can use in a search for one. 



borrowed and exmined copies of all available printouts generated 
for February 1974. There were a total of 278 printouts from the four* 
year college file (COL 4); 42 printouts of requests fran the two-year 
college file (COL 2); and 131 occupational file printouts available for 
eXOTilnatlon. The analysis of categories used was based on a total of 377 
searches (all files combined). The analysis of recency Is based on a 
total of 357 printouts (all files combined) * 
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For tha college files 'ne explanations of the ieparate categories 
groupings of charac Ua) are generally fairly full and clear* 

The layout of the Study Gui .^let^ however^ makes some categories 

lees visible or noticeable than others The COL 4 file contains about 
25 categories a^ 600 characteristics; the COL 2 file contains about 21 
categories and 350 characteristics. Each of the 1600 four-year colleges 
and 1000 two-year colleges in the respective data files are (theoretically) 
coded as having or not having each of the characteristics. 

The occupational file contains only 7 selector categories (consisting 
of about 80 characteristics) ; the descriptor categories consist of the 
sme 7^ plus 9 more* There are about 270 descriptor characteristics* In 
this file many of the categories have little or no explanation in the 
Study Guide, For example^ the category^ "Levels of Formal Education Usu-- 
ally Preferred or Required by ttm Employer^" contains 14 characteristics 
or gradations in mounts of education (from "less than high school gradua- 
tion" to "doctor's degree required or preferred"). The only explanatory 
statCTent in the Study Guide is "Several levels may be true of one occupa- 
tional title, "2 

The liaisons were asked to rate each category of characteristics in 
terms of several defined variables^ using three-point scales. For the 
college flles^3 the variables included the completeness of the categories 
and the output^ the accuracy of the output (including its timeliness)^ 
the clarity of the descriptive explanations of the category^ the 
appropriateness of the characteristics within the category^ the relevance 
or meanlngfulness to students of the Information in the category (regard- 
less of TSC's treataient)^ and the frequency of student uge of the category. 
For the occupational file^ liaisons rated the categories in terms of 
completeness (or eKhaustiveness)^ relevance, clarity of explanations^ and 
ToEicalness of the classification (l,e,7do the occupations listed as 
output serai to belong to the characteristics asked about?) The Appendix 
contains the liaisons' mean ratings of the college and occupational 
categories (Tables AS and A6^ respectively) , These tables give the TSC 
titles of the categories and also show the nmnber of characteristics In 
each category* 



ColleEe Files 

Some of the probl^s with any data file relate to how the decisions 
are made about what Information Is to be stored^ and how that information 



This has been i^roved in th^i 1976-77 edition* 

fuller explanation of this Important category seems warranted. The 
1976-77 revision of the Study Guide does not provide a more adequate ex- 
planation, 

^Since the two college files (COL 4 and COL 2) are highly similar^ for 
these ratings we did not ask Halsons to differentiate between the two • 

liaisons were asked to rate separately catagoriea included as selectors 
and categories us.d as descriptors* 
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is compiled. To collect and ass^ble Its college data^ TSC sends ques- 
tionnairea to colleges^ and makes "a strong effort to get as complete a 
return [as] possible* Data on [nonrespondents] are obtained from the most 
recent college catalogues and other reference sources...,*' Several pos- 
sible sources of error are obvious^ — t^o^ especially at large institutions^ 
are: Who fills out the questionnaire^ and how reliable is the Information 
furnished by that respondent? Definitions of category meanings may differ 
among respondents (e*g,^ what one college calls a prelaw or praned progrra 
another may not)* If Institutions do not respond^ then TSC must make 
inferences from the available reference sources, TSC acknowledges these 
as problem areas. Less obvious^ perhaps^ is the possibility of uninten- 
tional or purposeful distortion by the questionnaire respondent of the 
college's activities or requirements (e.g,^ describing the college In 
ways that encourage applications from particular kinds of students). 

Overall^ in terms of which colleges are included^ the liaisons 
indicated that both college files constitute a fairly complete inventory 
of institutions* The liaisons reported that a few undergraduate colleges 
were missing — e.g.^ SUNY at Old Westbury and C* W, Post.^ Neither of 
SUNY's ^o Health Service Centers (Upstate and Downstate) are Included^ 
both of which have third* and fourth-year undergraduate progrOTS. 

When asked if the COL 4 and COL 2 files comprised the total range 
of postsecondary educational institutions necessary for New York City 
high school students^ the liaisons agreed that it would be most desirable 
to Include selected graduate schools (e.g,^ Columbia University School of 
Journalism)^ specialized schools (e*g,^ New England Aeronautical Institute)^ 
technical^ trade^ and business sctools (e.g*^ Katharine Gibbs^ beauty 
culture schools)^ and other vocational schools where high school graduates 
could obtain skills training. 

Completeness can refer to the categories and characteristics chosen 
for classifying colleges in the flle^ as well as to the number and kinds 
of schools included. Overall^ the liaisons rated the completeness of the 
individual categories highly; all mean ratings were above the midpoints 
of the scales^ with"Costd' being lowest. They did suggest the inclusion 
of other categories^ such as slEe of college departments and the number 
of students in each^ percentages of minority groups on campus^ and high 
school course prerequisites for college majors. They also suggested that^ 
in the two**year college flle^ there should be a more refined breakdown of 
the Conventional Acadenlc Progrms of Study category. 

The liaisons' ratings of accuracy closely paralleled those for com- 
pleteness^ despite the fact that inaccuracies in the output were the 
subject of msBt of their day-to-day cotments. All means were above the 
midpoints^ with the lowest ratings given to "Costs^"^ g category that 
quickly becoi^s obsolete. Nevertheless^ liaisons noted that the costs 



C* W* Post is in the file^ but under its new nMe--Long Island Univer- 
sity^ C# W* Post Center--which Is not In common local use and which is 
not cross --referenced under the former nane, 
2 

"Costs" appears as a major category in the Study Guide^ and is made up 
of* (I) Annual Tuition and Fees; andC2) Annual Tuition^ Fees^ Room and 
Boards 
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are "in the right ball park." On a daily basis what proved most disturbing 
to liaisons was incomplete or mlsclasslf ied information* Several examples 
(from the 4 file) follow. Contrary to the information in the fila^ 
Queens College offers a communications major^ has a drama society (as of 
1973)^ and does not offer a major in dance. John Jay College was not (but 
now is, 1976) listed as having a major in law enforcament and corrections. 

While these two latter examples are illustrative of the problems of 
collecting and classifying infomation about large multi-unit university 
systems^ similar exraples were noted for smaller colleges and unlveritles* 
On the other hand^ liaisons frequently indicated that the college files 
contained information that they as college advisors were not aware of; 
after checking it independently^ more of tan than not CIS was correct. 

In certain categories more than others^ classification is a problem 
area. The School C liaison^ for examplej disagreed with how colleges were 
classified within the category of "Competitiveness" ; ^ she felt the clas- 
sification was quite at variance with her own experience- But what might 
be regarded as very competitive for one high school and its student body 
may not be as competitive for another high school with a dl^erent student 
body, 

CIS Is designed for a national audience* Obviously^ data bank clas- 
sifications cannot differ for each high school; however^ our analyses and 
the liaisons' ratings suggest that the stored information would be much 
more meaningful to our student population If It better reflected New York 
City experiences rather than^ as now^ national ones. 



According to TSC^ the data files are "updated on an annual basis in 
a major updating effort completed In the spring and early sunaner. Limited 
amounts of data are also changed In the periodic updating between the 
major updating efforts**.*" This statement leads to the expectation that 
In Period 1 (which began in February 1974)^ the college files should have 
been updated for the 1973-74 school year. 

Our examination of the February 1974 printouts showed a total of 639 
different four-year colleges retrieved from the file^ distributed all over 
the United States* Of the 639^ 9 colleges (1*4%) were updated in 1974^2 
22% in 1973^ 76% in 1972^ and 1 college (0*2%) in 1970, The two-year 
college printouts we exmlned contained names of 110 different colleges^ 
located in 19 states* Here the most recent infomation for one college 
was dated 1973^ and all others 1972, These findings do not support the 
TSC claim of annual updating, (The reader is cautioned to keep in mind 
that all these data are based on the very first month of the project. 
Similar analyses have not been done since.) 



"The category of Competitiveness contains 5 degrees of competitiveness. 
(There is some overlap between this category and that of Academic Charac- 
teristics of the Undergraduate Student Body*) 

^hen the computer prints the name of a college^ is also prints a single 
digit which represents the update (''vintage") year (although it is not 
labeled as such). For exmple^ a '-2" signifies that the Information about 
this college was last updated for the 1972-73 year. 
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The update patterns by state are not the srae as the total update 
pattern* In actual practice^ TSC updates its information one state at a 
time, We found that 83% of the four-year colleges in New York State had 
been updated in 1973 — far higher than the 22% we found nationally^ Quite 
early in the negotiations^ IWOE had suggested the desirability of TSC's 
giving priority to updating information on colleges in New York State; 
apparently they adopted this suggestion. 

Next we turn to an eKaminatlon of the frequency of use of the various 
categories in the college files. Measurement of use can be considered as 
one index of importance and relevance. ^ The categories of Majors and 
Location were employed in 98% and 81%^ respectively^ of the 270 searches 
of the QOL 4 file In February 1974. Coeducation and Competitivenesa 
ware used in two-thirds of the searches. The next two categories In rank 
order^ Size of Total Enrollment and Costs^ were used far less frequently 
(24% and 23%^ respectively). All other categories were used infrequently^ 
If at all; 11 of the 24 categories analyzed (one was inadvertently left 
out) were employed in 5% or Tewer of the searches* 

The use of categories in the COL 2 file differed slightly from that 
of the GOL 4 file. Generally^ somewhat fewer categories were used in the 
41 searches for OOL 2 information that we exmlned- of the available 
categories^ half were used in 5% or fewer of the searches. The most fre- 
quently used categories were Technological and Occupational Currlculmns 
leading to Associate Degree^ and as ±n COL 4^ Location. 2 (The category 
of Majors was probably little used because it is not specific enough to 
be meaningful,) 

These results strongly suggest that major course offerings and geo- 
graphic location are important to all students^ but that students' 
Interests in or requirMents for--use of--other infonnation varies with 
their educational alms. Based on these data^ COL 4 searchers exhibited a 
wider range of interests (used proportionally more categories) than 
COL 2 searchers. 

The findings with respect to usage agree with the results on the 
Initial Survey as to the factors students considered most important in 
choosing a college. Their first three choices on the Survey were the 
fields of study the college offers^ location, and tuition and other costs. 
The lowest Initial Survey -rankings were for the size of the college and 
Its accreditation statur.'' Thus our results with respect to frequency of 
use of college categories parallel to some extent the interests students 
showed on the Initial Survey, early in March 1974* It is worthwhile to note 
that, one year later (on a somewhat different mQasure, see Chapter VIlI) ^ 
students who had had CAG experience differed from non-users both in 



Another measure of relevance or meaningfulness was obtained from liaison 
ratings; overall the categories of infomation were rated lower on rele- 
vance than they ware on completeness, accuracy, and clarity of explanations. 
Ratings of appropriateness were similar to, but slightly higher than, 
those for relevance, 
2 

More than half (53%) of the two-year college searches asked about New York 
State, as contrasted with only 18% of the four-year searches. This finding 
might be interpreted to mean that for either costs or other reasons, two- 
year college applicants want to be closer to home. 
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respect to their knowledge of the meaning of accreditation, and in their 
estimates of the value of this type of info mat ion. 

The role of the liaisons In helping students select characteristics 
for use in a search cannot be ignored; many categories rated low in rele- 
vance by liaisons were the same ones least used by students (as shown in 
the Appendix^ Table A7) . We cannot be certain whether the liaisons- low 
ratings reflected student behavior or, whether liaisons' own feelings of 
reliance influenced student behavior. In Period 1 especially^ as the 
liaisons were learning how to operate GIS and establishing procedures for 
student use, a great amount of effort went into helping students focus on 
characteristics to use in a search. All schools^ as noted^ coped with this 
in various ways-*-all of which had the effect of limiting choice to one 
extent or another. School D, for exmiple, Included only certain charac- 
teristics in their modified guide; the School C liaison was observed to 
steer students away from characteristics that she felt were misleading* 
As the School A liaison expressed it, "we^ consciously or unconsciously, 
made choices [for the student], introducing our biases [l.a#, when we got 
no response from the student or had other knowledge that (s)he wasn't 
making appropriate choices] and limiting the decision-making aspect of 
the project^ one of its strongest points." 



0 ccupa t iona 1 F lie 

The structure and organization of the occupational file differs In 
many significant ways from the college files, Flrst^ as already mentioned^ 
there is a much smaller number of characteristics usable as input (7 
selecter categories vs. 25 COL 4, and about 80 characteristics vs, 600 
COL 4). Secondly^ this file is generally less easily understandable^ 
because the concepts underlying the GIS classification of occupations are 
less well known and sometimes seen to be at variance with conmon sense, 1 
A third major difference is that occupations are only sTOpled j selected 
and based almost completely on the infomatlon contained In the Dictionary 
of Occupational Titles ( D.O.T. ) , which was last revised in 1965. 

TSC provides no explanation of the basis for choosing their sample 
of 1^300 from the 20,000 occupations listed in the D.O.T. ^ and inspection 
of these occupations^ as listed In the User Manual, raises questions about 
the adequacy of the sfflnpllng. To axmlne tba situation, we checked to 
see whether the occupations of interest to students^ as stated on the 
Initial Survey^ were in the occupational file* 

Of the 341 occupations listed on the Initial Survey as first choice^ 
41% were in the GIS file, and 35% had one or several closely related 
occupations in the file; but 24% were not there (in Period 1)* Exffinples of 



Common sense suggests that a petroleimi engineer^ for example, should be 
classified either under Manufacturing (with other engineers) or under 
Marine Sciences (off-shore drilling)- This occupation can, according to 
TSC, be properly classified under Transportation since a petrolerai en- 
gineer is involved with transporttng oil over pipelines. 
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occupations not In the TSC system include: translator^ athlete (eKcept 
for jockey)^ acologlst, owner of own store or business^ member of armed 
services^ marine biologist (which was^ however^ In the COL 4 ETnerging 
Fields category)^ sanitation worker^ zoologist^ computer progranmier^ 
travel agent^ news broadcaster^ and model.! The liaisons felt that while 
the file was^ in general^ complete^ there were many omissions of very 
much Intsrest to students. Liaisons also indicatad areas of general 
omlsslon-^-not enough emphasis on spot ta-*related and artistic occupations^ 
and on occupations requiring two and four years of college^ That is^ 
liaisons felt there were too many jobs requiring a Ph*D* and too many 
'■immediate opportunities at the lowest level of anployment p*^ 

For the occupational categories^ the liaisons rated three of the 
same variables as for the college files -^- completeness , releyance , and 
clarity of eKplanatlons (see Appendix^ Table A6) . The average ratings 
of both the occupational selectors and descriptors^ in terms of complete- 
ness and clarity of explanations^ were clearly lower than those for the 
college flles^ and particularly so for clarity of explanations* The 
relevance of the occupational Information was rated^ overall^ just 
slightly higher than that of the college Information, 

This reinforces the Impression that the high schools do need access 
to good occupational Information. The liaisons generally gave lower 
ratings to the occupational selector categories than to the descriptor 
categories--especially for clarity of explanations and for the fourth 
variable^ the loglcalness of the classifications* Their Jud^ents agree 
with our own comments on the inadequacies of the occupational selectors^ 
which are^ of course^ crucial in determining the Job choices obtained in 
the printout, (The ratings of some of the separate selector categories 
will be discussed subsequently.) 

Unlike the college files^ the occupational file does not provide an 
update year. As part of each occupational description^ however^ the user 
obtains 3 to 4 nanes and addresses of organizations to which to write for 
further information (see sample description in Figure 2) . To estimate 
the recency of the occupational file data^ we used the nmnes and addresses 
listed on all occupational printouts for February 1974^ which yielded a 
total of 188 different job titles and 150 different sources (addresses) 
for additional information. 

We mailed letters requesting Information to the 150 sources. After 
two months^ only half had responded. Of the 75 nonrespondentSj 18 letters 
(requesting information on 27 occupations) were not deliverable, A more 



On the other hand^ the 1974 and 1975 versions of CIS contained redundan- 
clei--epg*^ steward and stewardess^ 11 kinds of draftsmen(aeronautlcal to 
structural)^ 5 geologists^ a blues and concert singer^ and so on. 

The 1976-77 edition of the User Instruction Manual includes sanitation en- 
gineer^ translator^ athlete^ and news broadcaster; the new description of 
marine biologist is exactly the sane as biologist. The other examples are 
still omitted. 
2 

When these letters were returned to us^ we were easily able to check in 
the telephone directory 5 sources with New York City addresses. Two were 
listed in the 1972-73 directory^ and the other three were In neither the 
1973-74 nor the 1972-73 directories* None of the addresses from whom we 
requested computer^related occupational information replied. 
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important fact ii that the 57 other nonrespondants^ who evidently received 
our letters (requesting inforDaation on 61 Jobs), did not reply within two 
months. These are sources to which many students send requests for infor- 
mation. 

The 75 organigations that did reply (covering 100 occupations) sent 
literature which we reviwed for appropriateness* Only 27 responses were 
judged as appropriate to the request-"that is^ we received what we sent 
for.l Overall, the sources of additional information suggested in the 
deecriptions are not adequate in terms of response rate nor appropriate- 
ness of descriptive materials sent, and many addresses are not up to date. 

In the OGcupatlMial file. It is relatively easy to determine the 
aKistence of a problOT, but very hard to Identify the reason for it* For 
exOTaple, if the list of occupations retrieved from the computer seems to 
have little relation to the Input, it is difficult to say whether TSC's 
classification was faulty (illogical)^ whether the D^O^T, 's schma is too 
compleK^ or whether the user misinterpreted the definitions of character- 
istics (inadequate or unclear explanations). The llalions' ratings of 
clarity of explanations of the selector categories were uniformly low^ 
eKcept for the category of 15 Occupational Clusters, Actually, the latter 
selector category is the only one that receives much explanation (In the 
Study Guide), and it is fairly self-evident; the other six categories have 
vary little or no explanation* ExOTaples of unclear terms are plentiful; 
one illustration from the Study Guide follows." Following the "Character- 
istics About Industries — Or Fifteen Occupational Clusters" Is the category 
"Characteristics About Occupations within Industries*" The only explana- 
tion of the latter category Is these "characteristics *. .are the nine 
Occupational Categories of the D.O.T. . Volime II*..," There is no explana- 
tion of how to use characteristics In that category either alone or together 
with the 15 ClusterSp 

Consideration of the loglcalness of the job classifications seems to 
Involve several related problems. Besides seming at variance with common 
sense, another source of confusion Is that the same occupation may be found 
using several characteristics In one category (Amount of Formal Education 
Required or Preferred is a good exmple), or may be classified in more than 
one Cluster. In the fall of 1974, TSC sent an addenda to the User Manual 
containing a list of all occupations in each of the 15 clusters* While 
this aided substantially, there are not such "keys" for certain other 
troublesome categories, such as Interests* For the Interests selector 
characteristics, the Study Guide tells the user to pick one interest (but 
not both) from a pair; yet no pair accounts for all occupations. The 
unfortunate conclusion is that many occupations in the data file are not 
coded on certain Interests pairs. Liaisons gave the lowest ratings to the 
Interests category on all variables except relevance. As we shall see. 
Interests ranked second highest in frequency of student use, so this is 
more than a minor problan* 

^Others said they would send materials if we prepaid shipping charges; in 
one or two other instances, there was a nontrlvlal charge for the materials, 
2 

Agaln^ the 1976-77 Study Guide attanpts clearer explanations, Including 
a glossary of terms. Some categories still remain fairly inexpllcable-- 
e.g.. Characteristics About Occupations Within Industries. 
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AnQthar relatad complication is that many classification "rules-- 
cause confusion- For eKample^ when a user specifies a desire for jobs 
at a piiticular educational level^ the list outputted yields those occu- 
pations where an onployer would either prefer or require that level of 
training. In many instances^ the result is that the jobs outputted 
usually require more education^ but the student may be unaware of this 
fact^ unless (s)he clearly understands the implications of the '^preferred 
or required" classification scheme. 

The general problm is that there is no set of explanations^ no "key^" 
that helps the user understand the aystmi underlying the classifications. 
It is often unclear why an occupation was included and on what basis 
(other than the D.O.T# ) it is classified. This is obviously a trmendous 
disadvantage in that it requires much time to become an "occupational 
file expert^" and even the rmBt expert cannot acquire a full understanding 
of the clasiif ication. The most expert tend^ rather^ to fi^re ways around 
the problem. 



The analysis of student use of the 7 occupational selector categories 
was similar to that done for the collage files. The aim was to obtain 
objective evidence on their importance or relevance. 

Since there are only 7 categories to select from^ the results are^ 
as eKpected^ that frequency of use of each occupational category was 
hi^er than for the college categories. Of the 66 searches eKamined^ 
97% used at least one characteristic from the category of 15 Occupational 
Gluetei^s83% used Interests^ 82% used Characterlst " bout Occupations 
Within Undustrles^ 73% used Levels of Formal Edu^ .d 55% used 

Aptitudes, In spite of the relatively small number of occupational 
categories and characteristics^ aljMst no one used Training Other Than 
Fomal Education (9%)^ or Special Vocational Training Time (6%)* Thus^ the 
effective range of occupational selectors was essentially limited to five 
categories; and even categories like Aptitudes and Interests^ which are 
not very satisfactory^ had to be used in half to four-flfths of the searches 
to narrow suf f Iciently , There were suggestions for additional categories; 
for eKample^ three liaisons suggested adding a category describing which 
high school and/or college majors are related to the occupation. 

The liaisons' ratings can aid in determining the reasons for the 
relatively low usage of some categories. They rated the Aptitudes 
category very low on all four variables^ suggesting that they considered 
this category^ which does not reflect the kinds of self -information that 
students possess^ as unsuccessful in providing meaningful Job choices. 
Part of the reason for the very low use of Special Vocational Training 
Tinie and of Training Other Than Formal Education may be that students are 
more concerned with^ or at least mora aware of, the existence of formal 
educational requirements for Jobs* But consideration of the liaisons' 
relakively low ratings of these two least-used categories (especially for 
clarity of explanations), as well as our own analysis. Indicates that they 
are not used prtoarlly because of the lack of any explanations* For 
exraple, (until 1976-77) the Study Guide does not define differences 
between ''apprenticeship," --on-the-job training," and "In-plant training 
other than on-the-job training." 
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Based oti our eKperience and the evidence collected^ it is our opinion 
that the Guidance Information Systan Is a good starting point for a com- 
puteriEed college and career information retrieval syst^. One of its 
major defects is that It is not responsive specifically to the needs and 
concerns of the New York City high school student population. GIB is a 
national venture* The major significant implication of this latter fact 
la that TBC^ the group prtoarily responsible for maintaining and updating 
the syst^^ is necesaarily more concerned with marketing and service 
problema than with educational ones* TSC has provided responsible service^ 
over and above what they contracted for. We feel It is of great Importance 
that educational principles should fom the basis for choices of what files 
the ay stmt should contain^ what information to include in a flle^ and how 
to categorize it# It is unrealistic to expect TSC to devote their time 
to developing a set of educational objectives and designing content and 
methodology that will facilitate carefully defined outcomes. Their proper 
concern was to design a syston that has widespread usefulness with diverse 
student populations across the country-^whlch they have done. 

In the 15 months of using GISj 4137 New York City students in five 
high schools obtained a great deal of benefits; an linproved system could 
provide additional ones. New York City students need a systCTi with some 
modifications^ some eKpanslons^ some unique additions^ and maybe some 
deletions* These necessary changes would take a substantial degree of 
f ftnpj commitment^ and for TSC a shift of purpose which is unrealistic to 
expect from a proprietary organization with an already useful and market- 
able product* 
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CHAPTER VI 
STUDENT USE OF COMPUTERIZED SYSTEM 



This chapter presents and discusses data on the afiwunt of student 
uiage of GAG during Periods 1^ 2, and 3^ In terms of the nimbers of stu- 
dent users, the total nimiber of times students used CAG ( uses ) , and the 
student users' grade levels* To fill out the picture^ we will also In- 
cluda data on aiwunt of use of the different ftles^ the proportion of 
time the student was present (direct) or not present (indirect or batched)^ 
and whether (s)he searched the file or requested a description of a college 
or an occupation from a file. 



Users and Uses 

Table 3 (page 54) shows^ under "Total Users/^ how many different 
students had CAG experience in each time period/ The "New Users" columns 
Indicate^ for each time period^ for how many of these different students 
CAG was a new experience — that is^ how many had not used CAG In any other 
period* In Period 1 the "Total Users" always equals the "New Users"; In 
Period 2^ however^ of the 954 total students served In all schools com- 
bined^ 708 had had no prior CAG experlence--lpe., were new users. And^ 
246 (954 - 708 ^ 246) students had used GAG in both Periods 1 and 2* The 
figures for Period 3 are to be treated similarly* During the 15 months 
under consideration^ 4137 different (new) students in the five high school 
used CAG to obtain college and/or career information; in addition^ 504 of 
these students used CAG in two periods (1 and 2^ 2 and 3^ or 1 and 3),^ 
Added together^ there were 4641 students who used the systOT 6789 times* 

The largest number of users and uses—about half the grand total-- 
occurred during Period 1, In Periods 2 and 3 usage was lower^ with the 
largest decrease In Period 2^ the Fall sanester. Looking at the indi- 
vidual schools In Table 3^ in every Instance use was greatest in Period 1 
and least In Period 2. Moreover^ the schools maintained their relative 
positions! In every period School E had the inost new users and total 
uses; Schools A and B ranked second or thlrd^ School D fourth, and School C 
continuously served the fewest students. 

Although there were actually fewer students served in Periods 2 and 3 
than in Period 1, how nnich of a real decrease occurred Is not apparent 
from the data In Table 3. There were differences in the lengths of the 
time periods (5, 4, and 6 months, respectively), as well as differences 
In the nmnber of on-line days* As we have already seen, the availability 
of extra days in Periods 2 and 3 made possible by the conmunlty college 



There were among this group a small nimiber who used CAG in three time 
periods; these few students were counted twice, thereby inflating the 
504 nimber somewhat. 
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TABU 3 

mm OF iTlTOENT USERS m TOm HER OF USES^ BY SCHOOL m M Til PERIOD* 





School 








Period 


k 


B 




D 


E 


All Schools 


Uiirs 




lliBti 




Users 


Usee 


Users 


Ussi 


Users 


Uiis 


Uiirs 


Usii 




Total 


Usii 


Total Not 


UiSi 


Total to 


Total New 




Total New 




Total New 


1 (5 nios.) 


497 


49? 


m 


445 443 


351 


383 383 


540 


392 392 


551 


557 557 


911 


ink im 


3332 


2 (4 wh) 


205 


165 


271 


213 173 


269 


61 42 


84 


84 53 


117 


379 273 


337 


934 708 


1278 










409 330 




SO 73 




193 174 




421 333 


739 


1413 1133 


2179 


3 (6 ios.) 


282 


243 


560 


329 


101 


230 


All Periode 




903 


1610 


- 978 


1349 


- 498 


725 


- 619 


898 


- 1167 


2207 


^ 4137 


6789 


(NuiiSlCAG 
iniorithan 
one pifiod) 


(19) 


(124) 


(33) 


(40) 


(228) 


(504) 



*Thi liril: time a student usid GAG (s)he was counted as a "New User "; any ripeited use by that sffise 
student in the time period (i)he first used GAG was added to the "Usii," If the sfflii itudent uild GAG in 
another tiie periodj (s)hi was counted is a "lotal User" in that tin piriod, 

In School large nimbers of studints were not listed on thi User Sign^In Sheets, In the other 
schools and in Periods 2 and 3^ there were only sniall nunibirs of unidintifled usergi Froni the TSC iuto- 
iitid sifflnaries^ wi could estiMti idditionil us^ in each echool, and countsd thiii le "New Uiers/' which 
may thirefori involvi soine duplicition. 
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projact resulted in a large increase of on-line days for the high schools 
and a decrease in mean hours of daily use* To determine what usage in 
Periods 2 and 3 might have been^ had usage continued at the Period 1 rate^ 
we could have predicted a total of 2747 uses in Period 2 Instead of the 
obtained 1278^ and 4726 uses in Period 3 instead of the obtained 2179. 
Altl^ugh this is the least conservative prediction^ 2 in both Periods 2 
and 3 use was approximately half of what could have been expected* 

Even among the highest users there was a decline in Period (s) 2 
(and 3)^ reflecting that both the initial l^petua to serve as many stu- 
dents as possible wore off^ and that use in the Fall semester might be 
lower than that of the Spring term* We also have some evidence that the 
schopls began to use the computer differently^ and perhaps with more 
benefit to students. We will return to this point in the data on direct 
and indirect usage. 

In connection with the results for the individual schools^ two facts 
need to be considered. First^ Schools C and D experienced the most severe 
cutbacks in staffing in Periods 2 and 3 which suggests that staffing has a 
aritloal influence on aoount of usage* Second^ Schools C and D are^ 
generally speaking^ the most aff luent--both with a strong college orienta- 
tion and a reputation for academic excellence; this suggests^ perhaps^ 
that a systaa like GIS may be least tised in similar schools. 



Table 4 (p. 56 ) is helpful in exmntnlng the schools' patterns of use 
during the three time periods. The upper third of the table presents the 
mean number of uses per on-line day. For all schools combined^ the 
results are as follows: Period 1^ 40.4 uses per day; Period 2^ 18. S; and 
Period 3^ 18.6* It can be seen that the average dally use decreased by 
slightly more than half in the latter two periods. For the most part^ 
this is in accord with the general decrease noted above^ but also reflects 
the fact that some schools-'-notably A and D--increased in the proportion 
of schaduled days usedi Thus the availability of extra days accompanied 
CAG use on more daysj but for fewer students per day. 

The mean number of uses per hour (and the average minutes per use) 
presented in the lower portion of Table 4^ tells us more clearly how many 
students' requests were being handled per unit of time on-line. This 
measure^ just like the mean uses per day^ was highest in Period 1^ except 
at Sclwol B (which shifted to mich more batched use in Period 3)* The 
differences anong schools and anong time periods^ however^ were not nearly 



See page 31j footnote 1« 

2 

In Period 1^ 3332 uses divided by 82.5 on-line days used (from Table 1) ^ 
40,4 uses per day* At this sttie rate^ for Period 2 there could have been 
(40.4 X 68 days used 2747 uses; and for Period 3 (40.4 x 117 days used «) 
4726 uses. This procedure does not consider the possibility that had there 
been no eKtra days available In Periods 2 and 3^ the schools could have 
been expected to use an equivalent proportion of scheduled days as In 
Period I (89%); with this correction^ the predicted usage would be 2409 
and 4063 In Periods 2 and 3^ respectively. 

66 

o 

ERIC 
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TABLE 4 

^ffiAN NUJfflER OF USES PER ON-LINE DAY A^ MAN USES PER TEroilNAL HOUR^ 
BY SCHOOL AND BY TM: PERIOD* 



Uses 


School 


8 


All 
Schools 


A 


"B 
D 


Li 


U 


£i 


Uses per Day 














Period 1 




34,4 


34.8 


34.4 


50.6 


40.4 


Period 2 


18,7 


19.2 


6.7 


9.0 


38.4 


18.8 


Period 3 


18* 1 






^ ■ ^ 




18.6 


All Periods 


9 ^ R 
» Q 




ID. D 


lo. o 


An 1 


25.4 


Uses per Hour 














Period 1 


8.0 


5.5 


7.0 


5.3 


9.2 


7.0 


Period 2 


7.5 


5.9 


5.4 


3.0 


7.7 


6.3 


f^erioo J 


7.2 


12.0 


6.3 


3.3 


6.3 




All Period e 


7.4 


7.4 


6.6 


4.2 


7.6 


6.8 


Minutes per Use*^ 














Period 1 


7 


11 


9 


11 


9 


9 


Period 2 


8 


10 


11 


20 


8 


10 


Period 3 


8 


5 


10 


18 


10 


8 


All Periods 


8 


8 


9 


14 


8 


9 



*Mean uses per day = Total uses = days used. These quotients were ob- 
tained from the respective figures In Table 3 and Table 1, Similarly^ 
mean uses per hour = Mean uses per day i mean hours per day (from Table 1) * 



nearest minute. 



go sharp as uses per on-line day* The schools tended to process roughly 
similar niOTbers of student requests per hour of use.^ 

Thus^ the decline In the anount of use in Periods 2 and 3--reflected 
in Che nmber of uses per day--did not affect the average length of time 
taken to process a student's request for information. The Individual 
schools did differ^ however^ in respect to the number of requests processed 
per hour. They tended to maintain the saae rank order^ with School E 
processing the most and Schools C and D the fewest requests per hour. As 
we will see below^ the primary variable to which these findings are related 
is the proportion of requests that were direct or batched. 



^ha overall slight Increase in Period 3 is entirely attributable to 
School B. 
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Before considering the data on diract»lndirect use^ we will consider 
multiple use of GAG by students. 



Multiple Use 

Soma students used GAG more than once^ and the proportions of 
"miltlple users" differed by school. Table 5 shows the data^ sunmed for 
the three time periods. Although a majority of the students used GAG 
only once^ over a third (38%) used it more than once* Using GAG twice 

TABLE 5 

NUMBER OF STUDENTS USING GAG ONCE AND MORE THAN ONGE^ 
BY SCHOOLj ACROSS ALL TmE PERIODS 
(Based on N - 4137 New Users) 



N Times 


School 


All 


GAG Was Used:* 


A 


B 


C 


D 


E 


Schools 


Once 


500 


666 


346 


422 


645 


2579 


ttore Than Once 


405 


282 


152 


197 


522 


1558 


% Using CAG More 
Than Once 


45% 


30% 


31% 


32% 


45% 


38% 


Breakdown of 
Multiple Use- 

2 times 


233 


202 


104 


138 


275 


952 


3 


93 


50 


31 


42 


119 


335 


4 


42 


23 


9 


13 


56 


143 


5 


28 


6 


7 


2 


34 


77 


6 


7 


0 


0 


2 


21 


30 


7 


1 


1 


1 


0 


11 


14 


8 


1 


0 


0 


0 


3 


4 


10-13 times 


0 


0 


0 


0 


3 


3 



*Total uses^ irrespective of different users^ equal 2579 + 2(952) + 3(335) 
+ 10(1) + 11(1) + 13(1) ^ 6789* 



was quite cOOTon^ and 128 students returned 5 or n^re times. In the in- 
dividual schools J the proportion of multiple users was about 30% in 
Schools Bj C^ and and went up to 457e in Schools A and E- 

Multiple use seems to be related to how CAG is presented* School A^ 
for example^ encouraged students to return with new questions or sug- 
gested areas to explore at the next CAG use- School E stressed as much 
coverage as possible^ so its studants needed to return for additional 
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laformatlon. School on the other hand^ Inlttally spent a considerable time 
with each atudent at the terminal^ exploring all options and alternatives; 
at School B there waa the smallest proportion of atudents using GAG more 
than once. The school^to-ichool difference probably reflects liaison 
practices and attltudei^ including methods of recrultoent^ conditions of 
use (e*g.^ direct vs* indirect; search va. description); it may also be a 
charaeteriitlc of certain types of students. 

The liaisons were asked tow they characterized students who requested 
repeated use of GAG. ^at "repeaters^" according to the llalsonF^ tended 
to be highly TOtlvated students^ or ones exceptionally f aicinat with the 
computer syst^. One liaison felt that the repeaters were students with a 
wide range of interests who could not decide on a single set of character- 
iatica for college or occupational information^ and who would continually 
return to explore different criteria. Another liaison indicated that in 
her eKperience^ repeatera exhibited a growing sophistication with the 
program^ using more refined criteria for searches^ and asking more specific 
questions about the output. Liaisons ccwmented that repeaters learned to 
use the system with much flneasej and began to ask much more specific 
questions about themselves as well as about GIS, 



Direct Vs. Indirect Use 

The CAG experience can involve either having the student process his 
own request for information^, being present at the temlnal while an oper- 
ator processes the request^ or having his requeat processed while he is 
not present. We have called the l^at condition of use Indirect or batched. 
The first two conditions--both involving the presence of the student--ia 
referred to as direct use; for all Intents and purposes^ there appears 
to be no difference between thm. ^ 

Table 6 (p. 59 ) shows the percentage of direct and indirect use for 
each school for each time period. During Periods 1 and 3^ for all schools 
combined^ only a little over a fourth of the students (29% and 28%) were 
present at the processing of their request; thus^ indirect use predominated 
In the Spring smesters. During Period 2^ however^ nearly half the 
students (48%) had direct experience^ and every school had as ^ch or more 
direct use than In Period 1. The surge in proportions of direct ^use In 
Period 2 was in response to IRDOE's feedback to the liaisons^ implying a 
change in this direction. 

In the individual schools^ the proportions of direct use differed 
from the overall averages. In Period 1^ Sctool B stands out as having 



There were actually very few Instances where students processed their 
own requests. Basically^ there were two reasons i the machine commands 
as presented In the Study Guide were too difficult for the average stu- 
dent to comprehend on his own^ and telephone security concerns as well as 
the need to establish an efficient operation discouraged most liaisons 
from attOTptlng this more frequently. In every school, however^ there 
ware individual students and student aides or monitors who operated the 
terminals ' on 
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6 

PERCMTAGE OF DIRECT AND INDIRECT USe/ BY SGI»OL AW BY TIME PERIOD* 

(Figures In Percentages) 









Time 


Periods 










Sehool 


1 


2 


3 


All 


Periods 




Direct 


Indirect 


Direct Indirect 


Direct 


Indirect 


Direct 


Indirect 


A 


29 


71 


56 


44 


25 


75 


32 


68 


B 


45 


55 


45 


55 


3 


97 


29 


71 


C 


29 


71 


46 


54 


88 


12 


39 


61 


D 


21 


79 


76 


24 


52 


48 


33 


67 


E 


25 


75 


39 


61 


37 


63 


30 


70 


All 
Schools 


29 


71 


48 


52 


28 


72 


31 


69 



*Uses that could not be identified as direct or indlract were not used in 
calculating the percentages. For all schools all periods combined thara 
was a total of 188 unidentified uses^ of which 135 were for School D in 
Period 3* 



conildarably more direct use (45%) than the other schools; it maintained 
this sane proportion in Period 2^ but shifted radically in Period 3^ 
when 97% of its use was indirect. School D in Period 2 had far more di- 
rect use (76%) than the other schools; and while this proportion dropped 
to 52% in Period 3^ D still remainad above the avaraga. School E had 
the largest proportions of Jndlract use In every period (75%^ 61%^ and 
63%); viewed along with the j.able 7 data^ these uses were mostly to 
obtain descriptions* School A had somewhat more than average direct 
use in Period 2^ but dropped off in Period 3* The School A data for 
Period 3 are misleading^ since most requests for deserlptlona of a career 
or college that followed a search were subsequently batched by the liai- 
son.- In other, words ^ many of the indirect uses in Period 3 were uses 
for students who had used CAG directly earlier in the period. 
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We have already noted (in Table 4) , that Period 2 had the fewest 
uses per unit of terminal time. We can now examine the findings with 
respect to direct vs. indirect use in relation to mean minutes per use^ 
shown in the lower third of Table 4. The correspondence between pro- 
portion of direct use (Table 6) and minutes per use is not perfect^ but 
comparing the respective entries in the two tables indicate that there 
is a relationship* That is^ the data generally conflm the impression 
that direct processing takes longer than indirect processing, Thus^ when 
the overall proportions of direct use went from 29% to 48% to 28%^ 
minutes par use went from 9 to 10 to 8 minutes; or when direct use In 



^ost counts in this report (on number of usea^ multiple uses^ file use^ 
and so oh) follow the sane procedural In general, any two requests by 
the same student on different days were considered as two uses ; two 
different requests by the same student on any one day were also counted 
as two uses. fj q 
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School B dropped from 45% to 3%^ minutes per use dropped from 10 to 5. 
(As anticipated^ there does not appear to be any relationship betoeen 
mlnutei per use and proportion of searches. See Table ?#) There la a 
slight suggestion from these data^ borne out by obser^^atlon^ that the 
schools tended to becoM t^re efficient over titae in processing requests^ 
whether direct or indirect. 

The consensus apaong the liaisons was that direct interaction (as 
wall as searching) was a preferable experience for the student* Why^ 
then^ did the schools often make sore use of batching? The answer^ In 
part^ is that any perceived advantages to the student of direct use had 
to ba balanced against t^m and dmand considerations. We saw that 
School C's use of batching decreased in each period^ as did its volume 
of use. When there are relatively few students' requests to process^ 
thare is little problem In having enough on-line tlTO available so that 
the students can be present (provided^ of course^ that there is sufficient 
staff time for supervision) . The primairy appeal of indirect GAG use la 
that it permits serving a greater number of students. Indirect inter- 
action was also sometimes used because of scheduling problems* it was 
often difficult to schedule students to come in on the school's on-line 
day* 

The advantages of indirect interaction do not make it the preferable 
technique^ The Period 1 Log Day results support concerns that had arisen 
during less formal observation* In numerous instances the terminal op- 
erator had to change batched requests for searches because the specifi- 
cations submitted by the student were Improperly writtenj or yielded 
"too many" qualifying options^ or no options. In these cases^ the operator 
generally made a ^ess at what to add or delete from the request In order 
to give the student what he probably wanted. The resulting printout may 
or may not have been useful to the students SOTetimes the operator did 
not guess or could not make any changes; In that case^ the request was not 
processed and the student was asked either to submit modifications or to 
return in person* 

The value of having students at the terminal is threefold. First 
are personal^^ootivatlonal benefits i Students enjoy being at the terminal 
watching the typing of the input^ and seeing the results type out auto- 
matically. Many studerts appeared so fascinated that they strained to 
read the information as it was being printed-"even though this speed was 
not terribly fast* Liaisons liked the opportunity for extended personal 
interaction afforded when the student was present* according to the liai- 
sons at Schools A and such discussions may be as useful as the computer 
information. 

The second type of advantage accruing from haying the student at the 
terminal is that Cs)he can make inmedlate decisions on modifications In 
his/her original request^ can pursue new Ideas^ and can discuss the output 
as it appears. Students can change their mind about what they want as a 



^^g Day results showed that in May 1974 (Period 1) the average time 
required to process indirect requests ranged from 3 to 6 minutes^ and for 
direct interactions^ from 7 to 16 minutes* 
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result of the infomation that that characterist Ic provided^ and can respond 
to something they see during the run to ask for dlffarent types of infor- 
mation. 

Finally^ direct interaction reduces the routine and often cmnber some 
paparwork involved In explaining the output to (or revising the input for) 
the stu' ant. When the student is present and professional staff is avail- 
able^ eKplanations and discussions could and did take place during or 
Iraiediately after the process. A major problem with the Indirect mode is 
that of orientation and followup--that ts^ what takes place before and 
after a request has been batched^ 

Thaoretlcally^ the advantages of having a student present could be 
sacrificed to some extent If It were possible to insure that every student 
could be counseled with respect to defining his input requlrOTients and 
would receive discussion of his output. All the schools tried to conduct 
scheduled orientations and individual or group follo\*mp^ with varying 
success. 

The liaisons reported that from 1% (School Period 2) to 50% 
(School 1^ Periods 2 and 3) of students who had had their requests batched^ 
returned at a later date on their own with questions on additional re- 
quests. If^ however^ the student did not come to collect his batched 
request^ no followup discussion was possible; in these instances^ the 
liaisons sent the printout to the student's homeroom class* The proportion 
of students with no followup was estimated for Period 1 as 25% at School 
10% at School 50% at School less than 10% „at School A^ and zero at 
School B. The schools were more successful providing followup in Period 2; 
an estimated 3% of batched requests at School and 10% at School E re- 
ceived no group or individual followup. 

It la difficult to determine whether students who had indirect GAG 
escperlence gained useful information. The data to be presented In 
Chapter Vlllprovldes indication of benefits to students who had this type 
of exposure^ despite Impressions that indirect Interaction is not nearly 
as satisfactory as direct interaction. What Is clear is that batching 
should probably not be done for students with little or low motivation-- 
those least likely to return to pick up their printouts It seems unlikely 
that students who had indirect GAG eKperience and no followup gained much 
useful information* 



Searches Vs. Descriptions 

Within either the occupational or the college flles^ GIB can be used 
in two types of ways^ namely^ for searches and descriptions* In the first 
type^ a search^ the user selects characteristics or qualifications that 



There were^ as noted^ problras in some schools in releasing students from 
classes during the regular school day* "passes" had to be written to 
eKcuse a student from one class^ and if the use occurred during a period 
change^ another "late pass" was needed* 
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are toportant to him, and derives a list of jobs or colleges that meet 
the requirMtenti he has set, (See Figure 1, page 15.) The second type 
of uie is purely descriptive; the user has a particular college or occupa- 
tion in mind and asks the computer to supply deicriptlve details from its 
file* (See Fi^re 2^ page 16,) 

Of course, both these capabilities can be enployed In a request, and 
In mDst cases that is what happened* First the user conducted a search, 
after which (s)he obtained a description of those Itons on the resulting 
printout that were of most interest*! xhe type of use that predominated 
varied with the school and the individual student. 

Table 7 shows, for each school, the proportion of CAG uses which 
Involved searches for occupational or college information. In February 
1974 and In Periods 2 and 3; 100% minus the tabled entry is the proportion 
of descriptions-only* (We did not attmpt to extract figures for March- 
June 1974 because they h^ to be obtained from an eximlnation of carbon 
copies of all prlntouts-**a tedious and costly process*) Based on liaisons 
estbsates and our observations, February 1974 was fairly representative 
of Period 1* 



PERCENTAGl OF USES INVOLVING SEARCHIS, FEBRUMY 1974 AND 
PERIODS 2 Am 3, BY SCHOOL* 





School 


All 




A 


B 


C 


D 


E 


Schools 


February 1974 


79% 


79% 


100% 


99% 


51% 


84% 


Period 2 


80% 


100% 


99% 


69% 


37% 


67% 


Period 3 


59% 


99% 


100% 


95% 


37% 


67% 


Total 
Identified Uses 


902 


856 


292 


462 


1366 


3878 


Total N 

Unidentified 


8 


5 


35 


14 


48 


110 



^Percentages are based on uses which ^^ere identifiable as aither searches 
or descriptlons-pnly. Of the 3988 total uses Involved, 531 were in 
February 1974 and 3457 in Periods 2 and 3. Of the Pebruary total uses, 
15% were unidentifiable; for periods 2 and 3, less than 1% were uniden- 
tifiable. Any search which was followed by a description within the sane 
use was counted as a search* We did not analyse March through June 1974 
because of the relatively great effort involved prior to the TSC automated 
data program. 



In Table 7, this type of use — I.e., a description following a search 
In the sane file (and during the same session) --has been counted as 
•'search." To qualify as a "description-only," a session had to be dewted 
to this activity alone. « r> 
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Msst schools used the search mode B^ch more frequently than descrip- 
tions^only, except for School 1^ which consistently made the least use of 
saarchea« School A used searches almost eKclusively throughout^ as did 
School D--except in Period 2* At School after Period 1^ almost all uses 
were searches. At School A In the earliest periods jabout four-fifths of 
the uies were searches; in Period 3^ this decreased somewhat^ to 59%, 

Another way to view the data in Table 7 is to consider the reverse 
of the tabled entries-- the percentage of descriptions-only. At School A 
the proportions of descriptions-only (21%^ 20%^ and 41%) were generally 
higher than at Schools and D, These School A figures^ however^ 

reflect the liaison- s practice of encouraging students to think about the 
Hat they obtained from a search^ and then to return to obtain descriptions 
of the n^st Interesting Items. ^ The increase In proportion of descriptions- 
only for School D in Period 2 reflects that school's Increased use of the 
occupational file (see Table 8^ p. 65)^ and the liaison's feeling that that 
file was too unwieldy to search. At School the high proportions of 
descriptions-only may be partly responsible for their results (to be pre- 
sented later) on the Independent measure of decision-making skills and the 
favorable attitude of students toward the computer as a source of college 
and caraer Information. 

School-to-school differences reflected not only liaison attitudes, 
but studanta* Interests as well. At every visit to the five high schools^ 
IRDOE staff observed students using GIS to test or conflm what apparently 
were decisions already made. One student at School after completing a 
search In a college flle^ exfflnlned the list of colleges and said-- -with 
akeptlclsm--*"! never heard of 'X' College, not that I would have applied 
there if I had." Another student (School E) came to the terminal asking 
for a description of "a lawyer and a plumber"; when he was asked^ he told 
us that his father wanted him to go into his business (plumbing)^ but 
that he was Interested In the law. There are numerous other examples of 
students using the computer to confirm choices made previously. It was 
aot uncoranon for students to request a college file search and look over 
the output to see whether indeed the college they wanted to attend was on 
the list: When it was^ the students' excitement was obvious; when It was 
not^ and the reason uncovered^ ^ the students were appreciative. 

Most liaisons agreed that the primary value of GAG was in helping 
students explore schools and occupations In relation to thelp own personal 
interests^ abilities^ and qualifications. All felt that using the system 



Because the description was obtained at another session^ to be consistent 
we counted it as a descrlption-'only^ having previously counted the search. 
2 

A student (School A) Included in her requirements In a search that the 
college offer a major in secretarial science . After examining the list 
of colleges that met her requirements she asked the liaison why "K" Collage 
was not on the list. By checking the data file first (and then the catalog) 
the liaison was able to tell her that "College X" did not offer her major. 
It turned out that the student had already been accepted by and decided to 
attend that college. As a result of this interaction the student changed 
her plans^ electing to enroll In a college that offered the field of study 
in which she was Interested* 
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to obtain deseriptlve infomatlon from a source as novel as the computer^ 
and with more Ijomedlacy and personalization (in the printout) than from 
library references^ produced enough student interest and Mtivation to 
make such use worthwhile. While this argianent has merit'-'^and is supported 
by ttie findings with respect to the impact on students — ^It seems to us 
that the unique feature cf GIS Is its search capabilities^ i*©,, its narrowing 
progrm format that shov the user the resulti of his/her own criteria 
In decision^'^king. 



Data Files Used 

Table 8 shows^ for each time period, the proportion of total uses of 
each file. In all sctools, tte proportion of use of the COL 2 file was 
small"a result in accord with student preferences indicated in the 
Initial Survey, ' We therefore combined COL 4 and COL 2 into a "College 
Total," which can be contrasted with the respective percentages of uses 
of the. occupational file, (In all periods together, there were only 33 
uses of the Scholarship file— less than 1%,) — . 

Regardless of the different priorities lEDOE suggested for Periods 
1 and 2, total uses for all schools combined were fairly evenly divided 
between occupational information and college information. In line with 
the priorities, there was slightly more use of the occupational Informa- 
tion in Period 1 (51%, vs, 48% college)^ and of college information In 
Period 2 (55%, vs, 44% occupational). In Period 3, when no priorities 
were specified, there was somewhat more use of the college than of the 
occupational files. 

The overall results conceal marked differences MOng the separate 
schools. Exfflnination of the results for individual schools in Periods 1 
and 2 suggests that most of thMi made some effort to comply with the 
priorities. In Period 3, however, use in Schools G and D was almost 
exclusively of the college files, by Juniors (from Table_ 9^. p, 67 ) * Of 
the Period 3 uses, about two-thirds of those at Schools A and E were in 
the occupational file, but less than a third of those at School B. 
Extrapolating from Table 9^ these School A and E uses (of the occupa- 
tional file) may have been largely by juniors; most of School B's use 
of the college files in Period 3 (70% of the total usei)^imst have been 
by juniors, since there were few senior users In that period* It is 
interesting that the big career Miphasis in School B did not lead to 
high use of the occupational file. Rather, the Interpretation may be 
that career Information needs at School B were being better met than 
college information needs. 

To cpnmant further On differences among the schools with respect 
to the balance between use of occupational and college information, 
School E usage remained the most constant throughout the three periods-- 



Use of COL 2 did increase slightly over the time periods-coverall, 
from 9% to 12% to 13%. Even in Period 3, use of the COL 2 file accounted 
for only 24% of the college file eKplorationsp 
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TABLE 8 

TOTAL USES OF EACH GIS FIU; AS A PERCENTAGE OF USES WITHIN 
EACH TmE PERIOD^ BY SCHDOL^^ 
(TOTAL USES - 6789) 
(Figures in Percentages) 



Files 


Period 


School 


All 


Total N 


Used 


A 


B 


C 


D 


E 


Schools 


Uses 


College 4 


1 




JO/o 


/ 1/e 


fi^7 


9 

ZD 4 


39% 


1297 




2 


54 


55 


91 


48 


24 


43 


555 




3 


24 


50 


87 


72 


29 


40 


875 




All 

Periods 


26% 


45% 


76% 


65% 


26% 


40% 


2727 


College 2 


1 


2 


12 


8 


11 


12 


9 


296 




2 


10 


15 


4 


14 


12 


12 


153 




3 


7 


20 


12 


19 


10 


13 


271 




All 
Periods 




167 


87 




1 17 


11% 


720 


Collage 
Total 


1 


20 


48 


79 


76 


37 


48 


1593 




2 


64 


70 


95 


62 


36 


55 


708 




3 


31 


70 


99 


91 


39 


53 


1146 


• 


All 

Periods 


31% 


61% 


84% 


78%a 


377a 


51% 


3447 


Occupa clonal 


1 


80 


52 


21 


23 


62 


51 


1705 




2 


36 


30 


5 


38 


63 


44 


560 




3 


69 


30 


0 


9 


61 


47 


1028 




All 
Periods 


69% 


39% 


16% 


21% 


62% 


49% 


3293 


Scholarship 
or Un- 
designated** 
















Schl.Unds, 


1 
2 


* 
* 




* 
0 


1 
* 


1 
1 


1 
1 


20 14 
8 2 




3 


0 


0 


1 


* 


* 


it 


5 0 




All 

Periods 


* 




* 


1% 


1% 


* 


33 16 



The percentages are based on the Ns for total uses In Table 3; e*gij of 
School A's 779 total uies in Period 1^ 20% Involved one of the college 
files and 60% involved the occupational file. 



*Lesi than 17m* 

**Students who signed in without indicating which filB(i) they used were 
counted as having one undesignated file use. Undesignated uses occurred 
mainly In Period 1 prior to automation of user Identification. 
O ' pro 
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tTO-thirds occupational and one-third college. School C almost abandoned 
use of the occupational file after Period 1 when the career counselor- 
liaiion becme inactive^ and School D did the sane iomwhat later but for 
largely different reasons. In both Periods 1 and 3, Schpol A made the highest 
use of the occupational file* 

There sems to be some inverse relationship between anount of use of 
the occupational file and the nature of the student population at the 
five schools* Schools C and the more affluent^ tended to use the occu- 
pational file least; Schools A and with a lower apcloeconomic population^ 
made greater use of occupational Information^ perhaps because fewer of 
their students will go on to complete college* 



Grade Level of Users 

To what extent did the sctools abide by the suggested priorities to 
stress use by juniors in Period 1 and by seniors in Period 2? And what 
happened In Period 3 when no priorities were specified? Table 9 (pA7 ) 
shows the results with respect to grade level; the table COTabines g : adea 
9 and 10 because there were very few or no 9di grade users involved* 

In Period 1^ 58% of the users in all schools combined were junlors-- 
nearly three tims the proportion of seniors (21%). In the Individual 
schools^ only School A had fewer junior than senior users (42% vs. 48%), 
Sctool as was mentioned/ did not have a full graduating class in 
Period 1^ and 42% of Its users were 10th graders. (6% were 9th graders *) 

In Period 2 there was a shifty In line with the priorities; the 
overall proportions are 56% seniors and 55% juniors. In Schools G and D 
the CAG users were almost entirely seniors. School B was the only school 
where senior users did not exceed Juniors. This was probably because so 
many Period 2 juniors knew about CAG as a result of the large n^ber of 
thm who were recruited in the sophomore year (Period 1^ Grade 10) • ^ 
School B had many more senior users in Period 2 than in Period 1 (51% as 
compared with the previous 2%)^ and use by 9th and lOfeh graders fell off 
sharply, from 48% to 8%. 

In Period 3^ the Spring tem of the 1974-75 school year, the schools 
made their own choice about grade level priorities* As Table 9 shows^ 
nearly three-fourths of all users were in grade 11; this Is a much higher 
proportion of juniors than in the ^o previous periods. Use by seniors 
was correspondingly lower, at 13%^ than in Periods 1 and 2. The only 
school with any appreciable proportion of senior users was School with 
30%. 

These results are not conclusive with rrfspect to the pre-established 
priorities, but they strongly suggest that the priority for use by juniors 



^There is some duplication In grade level; for an explanation, sea the 
footnote to Table 9* The highest proportions of grade 9 users were at 
School E--12%. 2%, and 9% in Periods 1 to 3^ respectively. 
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TABU 9 

PERCENTAGE OF GAG USERS IN G^ES 12^ 11^ AW 10 AW 9, 
BY SCHOOL AND BY TDffi PERIOD^ 
(Fibres in Percentages) 



period 


Grade 






Scnools 






All 


A 


B 


C 


D 


E 


Schooli 


1 


12 


48% 


2% 


12% 


25% 


14% 


21% 




11 


42 


50 


83 


69 


56 


58 




10-9 


10 


48 


5 


6 


30 


21 


2 


12 


-61%— 


— 41% - 




9b ft 








11 


35 


51 


o 


u 








10-9 


4 


8 


0 


4 


15 


9 


3 


12 


30% 


7% 


10% 


9% 


9% 


13% 




11 


69 


68 


89 


90 


70 


73 


10-9 

Total N with Iden- 
tified Grade Level 


1 
980 


25 
1002 


1 
494 


1 

596 


21 
1333 


14 
4405 


Total Unidentified 


4 


70 


37 


63 


62 


236 


Total N 


In Grade 9 


18 


29 


0 


0 


110 


157 



^The percantagea are baeed on the students with known grade level. This 
tabla involves some duplication becauie some students used GAG in more 
than one period. If^ for exanple^ a student used GAG in Period 1 as a 
tenth grader^ and again In Period 2 as an 11th grader^ (s)he is counted 
twice in this table* The Period 2 data involve 246 such duplications^ 
ai^ Period 3^ 258* 



in the Spring term accorded generally with needs in the individual schools. 
As we saw in Table 8^ however^ the priority for juniors- use of the 
occupational file in the Spring iemester did not accord with actual usage^ 
since over half of the Period 3 uses (by Juniors and seniors) were of the 
college files* There is more question about iroOE's priority for college 
file exploration by seniors in the Fall tera. Cross-comparisons would 
have been helpful in eKOnining how practical were the pre-established 
priorities. Did seniors in the Pall term. use mainly the college flles^ 
and in the Spring term did more juniors than seniors use the occupational 
file? Any such cross-tabulations would be very time-conauming, since 
they would have had to be done by hand* 

The liaisons were asked which students, in terms of grade (matura- 
tional) level^ benefited most and least from CAG* They tended to agree 
that the older students^ in grades 11 and 12^ got more out of the 
experience than did the younger ones^ although the liaisons at Schools B 
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and E itresied that CAG benefited high school students in all grades. 
When the liaisons were asked which studenta benefit most and least in 
teras of ability and aspiration level^ five of the six respondents agreed 
that students at or above grade level in ability^ the college bound^ and 
the highly laotlvated benefit the most* The rOTainlng respondent (School 
E) said CAG was of most benefit to those below average ability^ and of 
least benefit to the "eKtrraely bright." One liaison (School C)^ while 
agreeing that CAG benefits the highly motivated student^ felt that those 
with lower ability may be more receptive to the CAG method--i,e*^ to 
GIS*s decision -making programing fomat. 



Conalderlng the dlverilty of the analyses we perfomed with more than 
4^000 CAG users— ^ho used CIS with apparent benefits— there rCTialn several 
questions that need to be addressed^ but which are not answerable from 
the analyses we conducted* For axfimple^ the evidence presented in 
Chapter Analysis of GIS Data PileSj indicated inadequacies In the sys- 
tan; in particular^ the occupational file was less understandable^ less 
relevant^ and leas well-organised than the college flies. To what extent^ 
If any^ did the specific system ^ployed In this demonstration influence 
usage patterns? At Schools Aj C^ and the temlnal was located in (or 
near) the college office; at Schools C^ and E^ the primary liaisons 
were college advisors. To what extent did this predominance of collage 
emphasis bias the results? With the eitywide cutbacks In school staff In 
Periods 2 and 3 there was a reduction In the amount of staff time actually 
spent on CAG. Is the decreased use in Schools C and for exmple^ re- 
lated to the resultant college office workload^ and the fear of Increasing 
that workload through CAG use? Another consideration is the worsening 
economic conditions in New York City which reached a nadir in Period 3. 
Did this situation^ in fact^ influence use In School A? In Period 3^ 
School A's use of the occupational file by 11th graders may reflect stu- 
dent awareness that their plans for going to college might not be 
raalizab le. 



Despite the fact that--consldering all evidence tOiether--CAG is 
seen as providing some benefit to large numbers of students in all grades^ 
some priorities need to be considered and explored further. In terms of 
the kinds of decisions CIS requires^ we feel that most ninth graders 
might not be ready to use the system without a more structured framework 
In which to consider their options. The data with respect to tenth 
graders are inconclusive^ although the liaisons report that CAG causes 
students to begin thinking seriously about career and college decisions 
earlier in their high school years. Further study focused on these 
younger grades could reveal under what circrastances these students could 
profit from CAG. From the evidence we have gathered^ CAG can be success- 
fully used with juniors and seniors. GIS^ despite its limitations^ Is 
highly flexible and provides many different experiences for many types of 
students. In the next two chapters^ we will eKamlne the question of 
whether thoee experiences have direct Impact on student users. 
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COPTER Vll 
OUTCOMES: ANALYSIS 0F_ OOLLEGE APPLIGATIpNS 



One area In which wa anticipated differences between students with 
GAG eKperlence (uaers) and those with no GAG eKperience (non-ueers) was 
in their applicatiDns to college* Initially^ we reasoned that as a 
result of the computer's capability to retrieve all appropriate colleges 
in its dataflles^ there might be more applications by users than non^users} 
and that users who made college applications might apply to a greater 
number of different colleges. It also seemed possible that the acceptance 
rate for users might Increase as a result of their exposure to a list of 
colleges that fulfilled their requirmenta (i.e.j colleges that met the 
needs they had specified) . 

We could not corapare college applications of users and non-users* 
The closast we could approach comparative data was through an exmlnatlon 
of the college applications filed by the 1974 and 1975 graduating classes 
in the five high schools. The rationale for this comparison is that n^st 
of the 1974 graduating seniors (June 1974) had already completed the 
college application process prior to the Introduction of GAG in February 
1974* In contrast^ any uses oiE the GIS college files by the 1975 gradu- 
ating seniors either in Period 1 as juniors or in Period 2 as seniors, 
could have influenced their choices and actions. 

The reason that we could not compare users and non^users was that 
the records and recordkeeping procedures in four schools would have made 
collecting this data extremely impractical* Although Schools A and D both 
maintained the college application records in terns of individual atadents^ 
there were differences. At School a card was kept for each student 
listing the colleges (s)he had applied to; at School A^ however, a note- 
book was kept^ listtog chronologically the date an application was processed 
and for which student. Thus^ for School D It would have been possible to 
separate users and non-users^ and COTipare the number and kind of college 
applications; for School A It was not possible since there was no easy 
way to first compile a list of the colleges a particular student had 
applied to* 

At Schools and Ej records were maintained by college* Usually 

this took the form of a file card system of colleges applied to^ each 
college on a separate card^ listing the nBmm of the student (s) who applied* 
Similarly to School Aj it would therefore have been extremely difficult to 
separate users and non-users among the college applicants. 

Although we used the most suitable records available in the individual 
schools^ the figures in this chapter are less accurate than would be de- 
sirable for many reasons. One Important consideration is that the raw 
numbers of applications must be viewed in relation to the respective num- 
bers of graduates^ which we accomplished by calculating applications per 
graduate (APG) * The discussion In this sectlon**with the eKCeption of 
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CUNY— pertains to college applications rather than applicants. Other 
QompllQatlng factors are discussed below* 

To collect the college data, IRDOE staff visited each high school 
several times in June of each year- in SeptCTiber 1974 we visited again to 
re-check the June 1974 data, but did not do so for 1975 data, since 
the project contract ended in June 1975* The schor college offices 
process all applications and keep all records. They are supposed to be 
notified of CUNY admissions' they are not promptly and systematically 
notified about accaptancas or rejections by other collages. 

We analyaed the data within three categories, namely City University 
of New York (CUNY), State University of New York (SUNY), and' other'"collega 
the latter two make up the non-CUNY total. Applications to CUNY represent 
just one per graduate; those to SUNY and those to other colleges may rep- 
resent more than one application per graduate. There is undoubtedly 
overlap OTong these three categories of students-who^applied* Our taprea- 
slon when collecting the data and from dlicussions with the college 
advisors was that most students who applied to SUNY and/or"other"collages 
also applied to CUNY- Moreover, In at least 3 of the 5 schools every 
graduate was directed to file a CUNY appllcatlon--whether or not (s)he 
was going to go on to college. The advisors and students thliAlng about 
college regard this as a safeguard, since the graduate could be sure of 
acceptance by CUNY* 

With the advent of the Open Admissions policy at CUNY, in which every 
New York City high school graduate is admitted to one of the twenty units 
of CUOT, the student files only one application to the systCTa, on which 
fs)he expresses his/her first six preferences for particular coll :s 
within CUNY. Although acceptance to the system is guaranteed, thei is 
no guarantee that a student will be accepted to the achoola of hie/her 
choice* 

OTNY applications go through a central processing, and lists of 
acceptances are received by the sending high schools | these lists of CUNY 
acceptances state to which CUNY college the student Is admitted. We used 
these lists to determine the numbers of students applying (and accepted) 
to CUNY. If a student had applied but had not yet been listed as accepted^ 
our data would not show that CUW£ application. Neither do we know whether 
the accepting CUNY college was toong the student's preferences* Although 
CUNY notifies the sending high schools about acceptances, its applications 
processing differed in the two years. OTNY acknowledged late applications 
by June 1974, but had not yet done so by June 1975, when all our data 
collection ended. For this reason we probably Identified fewer CUNY 
applications in 1975. 

The colleges comprising the SIMY systCTi,^ and all other public and 
private colleges each require a separate application. It is easier to 
fill out an application to CUNY than to most other colleges, because the 
kind of Information CUNY requests is simpler and less detailed. It is, 
moreover, less expensive to apply to CUNY; other colleges require a fee 
to be paid when the application Is submitted. Finally, high schools 
differ with respect to the number of non-CUNY applications they encourage 
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a student to filej since each separate application requires a copy of the 
high sehool transGrlpt^ letters of reconmiandation^ and other information 
that the sending school processes . Thus, in order not to swrap the high 
sclKioiaV college office^ there is an unwritten limit placed on the number 
of applications the average student files. 

Our data with respect to non-CTOY applications comes from the 
records, daacribed above, which are maintained by the college office* 
Information about acceptances by non-CUNY schools is also maintained by 
the college office, but differs from CUNY acceptances. In general, 
many SUNY colleges notify the sending high school and the individual 
applying about acceptance; the high school does not receive SUNY 
acceptances (and rejections) until after the end of June^ so again we may 
have identified fewer of the SUNY acceptances in 1975, Accf?ptaace/re- 
jection letters from the other colleges go to the student directly, and 
often not- until May or even June; the student, in turn, may or may not 
notify the college office of the outcome. There la no apparent reason 
why the procedure with respect to these 'bther'fcolleges would affect counts 
of applications or their outcomes in the two years differentially, although 
the strong probability is that we did not uncover all data with respect to 
uther colleges In either 1974 or 1975, 

To smniariiei Because of college and high school recordkeeping 
yrocedures, the applications and acceptances for CUNY are fairly accurate — 
although better for 1974 than l975--while for SUNY and other colleges, our 
records reflect applications made (at least those listed by the high 
schools)^ especially for the 1974 graduating class^ but information about 
acceptancea/rejectlons was not always available. We regard all the ap- 
plications data to be discussed here as underestimates. 

Certain other factors affect either all schools, or particular ones* 
Few college offices noted whether the student sought financial aid, which 
would have been very useful information. For School C, records Included 
both January and June graduates, so the raw nwnbers of applicationsand 
of graduates are quite large (but this is controlled by the calculations 
of applications per graduate); the other sctwols Included only June 
graduates. Since School B, the newest school, did not have a full, 
official graduating class until 1975, its 1974 figures and some of those 
for 1975 are based on very small Ns, and Its percentages are to that 
axterit much less reliable than those for the other schools. This reduces 
the significance to be attributed to fluctuations from 1974 to 1975 in 
School B, and the discussion will generally Ignore such fluctuations. 
Also, iDOBt non-CTOY information from School B was not docimented, but 
feflects the*'recollections''of the college advlaor. For School A, the 
non*CUNY figures were modified to Include applications to colleges that 
appeared on lists of acceptances/rejections but for which we found no prior 
record of application. In these relatively few initances, we made an 
arbitrary asslgtinent of one application. 

One other background factor may have had very great significance-- 
a factor we could not have anticipated In advance"-namely, the serious 
worsening of general economic conditions between June 1974 and June 1975* 

^Flpjires from the Bureau of Labor Statistics indicate that in this period 
the national unemployment rate Increased from about 5*5% in June 1974 to a 
peak of atout 9% in May 1975, Unsnployment in New York City was much 
higher than the national averages, gn 
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Because o£ these conditions^ 1975 graduates might have been more likely 
to cliooia colleges where they could expect to get scholarships or other 
financial ald^ or to choose public colleges^ or if necessary to choose 
colleges (public or private) in the City or those to which they could com- 
route. Many students may have elected to apply to fewer private schools 
having expansive application fees* Still others^ who may have been faced 
with a tight Job market^ may have decided to go to CUNY* Concerning 
these factori we have no data; their possible effects can only be taken 
into account Jud^entally (if at all). On the other hand^ we do know 
that the economic conditions affected staffing in the public high schools; 
with the reduction In college office staff (most severe at School C but 
in evidence at School D as well ) , fewer applications might have been ex^ 
pec ted to be processed* Students may have been influenced^ directly or 
indirectly^ to apply only to colleges to which they had a reasonable ' 
chance of being accepted. 



Keeping all these qualifications in mind^ we can now turn to the 
results* Table lOj page 73^ preients the basic data* From 1974 to 1975 
there was an overall Increase of 13% in the ntEpaber of graduates^ in spite 
of decreases at Schools C and D* At the Bsmm timej there was an overall 
decrease of 9% (from 5366 to 4899) in the total number of college appli- 
cations* This decrease in total rnmber of applications was almost entire- 
ly attributable to School C^ whose graduates dropped by 3% but whose 
applications dropped by 21%, School like School C^ has a large 
proportion of college-bound students^ but from 1974 to 1975 its percentage 
drop In total applications was ataost the same as its drop in graduates 
(13%)* At Schools A and the number of total applications rmained 
quite constant in the two years^ but the number of graduates increased^ 
respectively^ by 6% and 27%* Thus^ at least in School the nimber of 
applications did not rise in proportion to the increase in the ntmber of 
graduates* (School B increased on both counts^ as expected,) In view of 
all these differences^ it is more meaningful to examine appllcatlona per 
graduate (APGs) ; these figures in effect cancel out variations In nmnbers 
of graduates and ni^>ers of applications* 



Applications per Graduate (APQb) 

liOoking first at the total applications per graduate In the middle 
of Table 10^ we see that overall there was some decrease in total APGs 
from 1974 to 1975 (from 1*5 to 1,2 per graduate)^ and this was the case 
in every individual school except Schools A and which rraained about 
the same* The decrease was largest at School G* In both years^ the 
individual schools mf.intained the sme rank order; School G (despite Its 
decrease) had by far the most APGS| then Schools D and E in an Inter 
mediate position^ followed by Schools and B with the least. 

Next we can inspect the bre^dowi of these totals into CUNY and non- 
CnJNYj and of the latter category into SUNY and"other" colleges^ to see 
whether the trends found in the totals also held true within these 
categories^ That is^ in 1975 as compared with 1974^ was there also a 
decrease in APGs to CUNY, to SUNY. and to other colleges? 
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TABLE 10 



wmi OF mmn m un m mm coliige applications^ inclodino ai other mmu, 

FOR 1974 ('J4) m im ('?5)^ M SCfflOL 



School 



A 



m '15 



B 



'74 '75 



C 



'74 '75 



D 



'74 '75 



E 



'74 '75 



All 
Schools 



'74 '75 



N Sraduatis 
M Applicfltions ; 
" Total N Idsntified 



NQn-CIM 



Other Colleges 



1070 1134 

980 984 

525 567 

455 417 

76 92 

379 325 



200 650 

141 33B 

Hi 257 

30 81 

8 26 

22 55 



982 953 

2415 1917 

907 657 

1508 1260 

821 630 

687 630 



900 780 

1322 1157 

519 480 

803 677 

450 393 

353 284 



386 490 

508 503 

180 213 

328 290 

102 111 

226 179 



Appllcatlgng per 
Graduate (APGi) 



CUNY 

Non-CUNY 



SUNY 

Other Collegis 



.92 .87 

.49 .50 

.43 .37 

.07 .08 

.36 .29 



.71 .52 
,56 .40 
.15 .12 



! BBS 



.04 .04 
.11 .08 



2.46 2.01 

.92 .69 

1.54 1.32 

.84 .66 

.70 .66 



1.47 1.48 

.58 .61 

.89 .87 

.50 .50 

.39 .37 



1.32 1.03 

.47 .44 

.85 .59 

.26 .23 

.59 .36 



3559 4026 

5366 4899 

2242 2174 

3124 2725 



1457 1252 
1667 1473 



1,51 1.22 

.63 .54 

,SB .68 

.41 .31 

.47 .37 



Applications Within 
Categories As t of 



ona 



ClY 



SUNY 

Other CoUegas 



531 581 
n 421 

391 331 



791 7i 

211 241 

61 81 

151 161 



381 347, 

m m 

341 WL 

281 331 



391 411 

611 591 

341 341 

271 251 



351 421 

m 581 

201 221 

451 361 



Nstij See text for fflany quilifieationi ^irtlinlng to thist flgurii. 



421 44X 

581 561 

271 261 

311 301 
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Since students make only one application to GUNY^ the CUNY APGs in 
Table 10 are aqual to the proportion of graduates who applied to CUNY. 
(This interpretation does not apply to the non-CUNY APGs.) Thus^ the 
"All Schools" column indicates that in 1974^ 63% of the graduates applied 
to CUNY^ and in 1975 this apparently decreased to 54%, 1 The rank order 
of the separate schools was similar to that for total APGs* School C was 
at the top in both years^ in spite of a sharp decrease in CUNY applications 
(92% to 69%); School D raided second (58% and 61%)^ and (ignoring School B) 
School E ranked the lowest (47% and 44%), The overall decrease in CUNY 
APGs from 1974 to 1975 was attributable to the decreases at Schools C and 
B (the latter went down from 56% to 40%). There was essentially no change 
in CUNY APGs at Schools A^ and 

In terms of non-CUNY APGs^ overall there were slightly fewer APGs 
to SUNY than to other colleges in both years-^a somewhat unexpected result, 
since SUNY would probably be less eKpenslve than the other colleges. Aa 
with CUNY APGs, both SUNY and other colleges' figures decreased from 1974 
to 1975 — SUNY from *41 to .31 and other colleges from .47 to .37. In rank 
order of individual schools. School C was again at the top in APGs to both 
SUNY and other colleges; Sctool D was-^roughly^^-next, followed by School E. 
Schools A and B were at the bottom. Almost no students at the latter two 
schools applied to SUNY (in either year)^ although School A students did 
apply to other colleges. 

The bottom third of Table 10 presents the data in a different but 
perhaps mote, easily comprehensible way. Here variations OTiong the schoola 
in total nimbera of graduates are ignored. The percentages express the 
number of applications made in a given year to CUNY, SUNY, and other 
colleges as a proportion of all applications made in that year . In the 
individual school i and in "All Schools," these figures look remarkably 
stable from 1974 to 1975. Viewing the data in this way, the implication 
is that the specific high school a student attends greatly influences the 
pattern of colleges (s)he will apply to, CUNY^ the least costly, is the 
TOSt popular at Schools B and A, while applications to non-CUNY colleges 
predominate at Schools C^ D, and E, In terms of this measure, the biggest 
change was In the proportion of other college applications at Schools A 
and Es , 



The overall decline in APGs from 1974 to 1975 does not sscti attrib- 
utable jco any school-related or student-related causes. The decrease in 
APGs til CUNY most likely reflects a difference in central processing of 
CUNY applications for 1974 and 1975, and the fact that we were unable to 
identify as many of the CUNY appllcmts in the latter year. Thus, as far 
aa the data with respect to CUNY is concerned^ we cannot be certain that 
there was really a decrease at all. 

Assuming that CUNY applications did not really decrease, general 



^eep in mind the limitations witii respect to the CUNY APGs for the 1975 
graduating clasa--particularly the fact that we probably Identified fewer 
CUNY applications in 1975. 
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econoffilc conditions might account for the decline In APGs to SUNY and 
other colleges* There are two ways economic circumstances might have 
affected the number of collage applications, First^ many students may 
have decided either not to go to college^ or might not have been able 
to fill out as many non-CUNY applications as they would have liked. 
Another poesible factor is that the overtaKed college office staff might 
have discouraged students from applying to any but the most realistic of 
their choiees. This latter reason probably accounts for the large 
decrease in APGs at School C (which^ in turn^ greatly influenced the 
"All Schools*' figure)^ but cannot explain the very large decline in APGs 
to other colleges at School E (which had essentially the same college 
office staffing in both years). 

We also examined the number of different colleges applied to^ both 
within each high school and across all high schools* Deaplte the overall 
decraase In APGi to SUNY colleges^ the number of different SUNY colleges 
applied to Increased from 31 in 1974 to 49 in 1975. (This is a 39% 
Increase^ greater than the 13% overall Increase in the number of graduates,) 
Students may have learned more about SUNY through GIS^ as well as from 
college advisors who were Increasingly studying and suggesting the SUNY 
system. To some extent this lends credence to our argument that economic 
conditions may play a decisive role in the 1974 and 1975 data^ since SUNY 
schools are generally less expensive than others. 

In both years^ students applied to a tremendous variety of colleges^ 
and no simple summary generalization can encompass the range* In 1974^ 
a total of 385 different non-CUNY institutions were applied to* As with 
the total number of applications^ there was a decrease in 1975 to 331, 
The only sizable drop was in School E* As implied abovcy the decrease 
was In the number of different other colleges* 

In one other analysis^ the non-CUNY applications were broken down 
Into those to two-year and to four-year colleges. These proportions 
tended to remain consistent from 1974 to 1975, An overwhelming majority 
of all applications went to four-.year colleges--927^ In 1974 and 90% in 
1975 # This finding agrees with students* preferences as expressed in 
the Initial Survey^ and with their low use of the COL 2 data file* 



Because of the many qualifications concerning these data^ the pos- 
sible Influence of CAG on the 1975 figures as compared with 1974^ 
becomes a speculative matter; our Impression is that economic conditions 
may have bean the most important factor. As we will see belowj anecdotal 
evidence exists suggesting that use of the GIS college files did Influence 
studant choices* It Is not an unlikely interpretation--although the one 
most favorable to this pro ject--that the CAG experience served to help at 
iaast some students make fewer^ more realistic choices. 



Outcomes of Applications 

Table 11 on the following page presents the outcomes of non-CUNY 
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TAB^ 11 

OUTCOMES OF NON-CUNY APPLICATIONS* 





School 


All 




A 


B 


C 


D 


E 


Schools 






/ J 


•74 


•75 


•74_ 


•75 


'74 


•75 


♦74 


•75 


'74 


'75 


Applications 


455 


417 


30 


81 


1508 


1260 


803 


677 


328 


290 


3124 


2725 


% Accepted 








Outcomes In 


Percentages 










/ 0/e 


74% 45% 








H'JL/e 






51% 55% 


% Rejected 


4 


11 


13 


17 


4 L 


1 Q 




12 


22 


12 


17 


15 


% Walt-Listed 


0 


1 


3 


1 




A. 


•J 




o 


n 
u 


3 


3 


7s UnknowQ 


55 


12 


10 


37 


23 


22 


28 




20 


33 


29 


27 


SUNY (for Ns^ 
see Table 10) 
U Acceptaa 


























55 


90 


88 


50 


50 


58 


55 


42 


43 


56 


52 


55 


Re J ec tea 


13 


8 


0 


23 


20 


14 


17 


10 


31 


16 


19 


13 


/o wait-LiScea 




2 


12 


4 


4 


5 


3 


3 


3 


1 


3 


4 


% Uoknovm 


30 


0 


0 


23 


26 


23 


25 


45 


23 


27 


26 


28 


Other Colleges 


























(for Ns^ see 


























Table 10) 
% Accepted 


38 


72 


68 


42 


55 


53 


49 


41 


62 


54 


51 


54 


% Rejected 


2 


11 


18 


15 


23 


23 


16 


14 


18 


9 


16 


17 


% Walt -Lis ted 


0 


1 


0 


0 


3 


3 


4 


3 


1 


0 


2 


2 


% Uid^nown 


60 


16 


14 


43 


19 


21 


31 


42 


19 


37 


31 


27 




























N Accepte 


d As 


% of Applications With Definite Outcomes 




(N Accepted Divided by 


[N Accepted Plus N Rejected]) 






Total Non-CUNY 


92 


88 


85 


72 


71 


75 


76 


78 


72 


82 


75 


79 


SUNY 


81 


92 


100 


68 


72 


80 


76 


80 


58 


78 


73 


81 


Other 


96 


87 


79 


74 


70 


69 


75 


74 


78 


86 


76 


76 



*See text for qualifications pertaining to these figures* 



applications '-applications to SUNY and to all other public and private 
colleges and universities* Included in the table are the percentages of 
applications accepted, rejected, and wait-listed, as well as the percen- 
tage with outcome still unknown at the time of data collection. Regard- 
leas of the number of applications flled^ if use of CIS caused students 
to apply to colleges that better matched their interests and qualifica- 
tions^ the acceptance rate might be eKpacted to Increase* 
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Before examining this hypothesis^ it is Important to note that in 
both yeara there was a high proportion of unknown outcomes--over a fourth 
of the total*! This fact severely limits any conclusions. For all 
schools CQmbinedj the proportion of unknown outcomas was nearly constant 
jErom one year to the next; in the individual schools^ however^ the propor- 
tion of unknown outcomes fluctuated inconsistently,^ It cannot be assumed 
that the unknown outcomes would be distributed in the same proportions as 
the known outcomes^ since students who were rejected would be less likely 
to inform their college office than students who were accepted. The sug- 
gestion is that if Bi\ outcomes could be known^ then we would find fewer 
non-CUNY acceptances. 

To arrive at a clearer Interpretation^ we looked only at applications 
with a definite outcome- -either accepted or rejected, (The percentage of 
students wait-listed was negligible in all instances,) The bottom of 
Table 11 shows the number of applications accepted as a percentage of the 
number accepted plus the ' aber rejected. For all schools combined^ about 
three-fourths of the non-C^NY applications were accepted^ and there was a 
slight increase in 1975^ from 75% to 79%. This increase is entirely at- 
tributable to an increase in acceptances of SUNY applications^ from 73% to 
81%; there was no change in the proportion of acceptances by other colleges. 
Ignoring School B because of small Ns^ the only sizable changes at the 
Individual high schools were an increase at Schools and C in SUNY 

acceptances^ and at School E in acceptances by other colleges* School A in 
1975 eKperienced a decrease in acceptances by other colleges. 

The Increase In the proportion of SUNY acceptances (in 1975) at 
School G coupled with the decrease in the number of SUNY APGs may indicate 
that rewerj but more realistic^ applications had been filed ,3 The increase 
in SWY acceptances for Schools E and A were not accompanied by sizable 
decreaaea in Apna^ but may also reflect more appropriate student choices. 
No clear school-to-school pattern emerges in exmiining the proportion of 
acoeptances to other collegeSp We cannot account for changes at Schools 
A and E; from examining School A's increase in percent accepted at SUNY 
and the decrease in their proportlDn of known acceptances by SUNY (in 1975)^ 
we suspect that the apparent decline reflects improvements in recordkeeping 
for 1975, The least accurate records kept at School A in 1974 was the entry 
of notices of rejection. 

Thua^ In conclusion^ the overall increases from 1974 to 1975 in the 
proportion of acceptances to total applications with known outcomes 
reflect changes with regard to SUIff, Due to the many qualifications in 



This is in spite of the fact that we re-examined school records in 
September 1974, 

^In School A in 1974, for example, 55% of the non-CUNY applications had 
unknown outcomes as compared with only 12% In 1975* For School A this was 
largely a result of new procedures adopted by the college of flcej In 
response to feedback from IRJDOE concerning 1974 findings. 

As Table 10 illustrates. School G had^ in both years^ a greater number 
of SUNY APGs than any other high school. A decrease from .84 to ,66 can 
be a chance fluctuation. 
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thes© data^ we can only make the guarded statement that CAG may have 
contributed to the filing of more realistic (i.e-^ SUNY) applications. 



The analysis of college applications was made in an attempt to as- 
certain whether the CAG experience affected college choice among students 
using the syitra* Specifically^ we anticipated that GAG use would increase 
the total number of college applications filed^ the diversity of colleges 
applied to^ and the rate of acceptance. Because of the incompleteness 
of the data and the numerous attached qualifications^ doubt is cast on any 
firm conclusion. 

In 197 4 as compared with 1975 there were mora applications filed ^ 
and more total APGs; we^ however^ looated fewer of the coBipleted ap- 
plications in 1975 than 1974. On an individual school basts (tgnoring 
School B)^ total APGs declined at Schools G and but remained about the 
same at Schools A and D* GUNY APGs dropped sharply at School G in 1975^ 
while they remained fairly constant in Schools A^ and in 1974 and 1975, 
half or more of the graduating classes at each school applied to GUNY* 
In both years there were fewer total APGs to SUNY than to the other public 
and private colleges; Schools C and D tended to consider SUNY more fre- 
quently than did Schools A^ Bj and--to some extent--E, GUNY was most 
popular at Schools A and Bj non-CUNY colleges at Schools G^ and E. 
This suggests that the high school a student attends greatly influences 
the colleges to which (s)he will apply. 

The data with respect to whether students applied to more different 
colleges in 1975 than in 1974 indicates that (taking the number of gradu- 
ates into account) Schools Cj and A applied to approximately as many 
different colleges^ although School E applied to fewer* There was an 
overall increase in the total number of different colleges applied to 
within the SUNY systOT in 1975. 

The acceptance rate would be expected to increase in 1975 if students 
(many of whom had GAG experience) had applied to colleges that better met 
their qualifications and Interests. Again ignoring School and looking 
at the findings only with respect to known application outcomes^ there 
was essentially no significant change with respect to other colleges^ but 
there was an overall increase in acceptances to SUNY--at School A^ an 11% 
increase; at School G^ 8%; at School 4%; and at School E there was a 
20% increase in SUNY acceptances in 1975* 

In sumary^ use of GIS may have ameliorated decreases In college 
applications that would otherwise have occurred because of severely 
worsened economic conditions in 1975 | or the observed decreases could 
mean that GAG brought about more realistic^ and fewer^ appllcationa. For 
SUNY in particular, the decrease in APGs, together with the increases in 
number o^ different SUNY colleges applied to and the proportion of known 
acceptances, favor the latter interpretation. 

There are better tests of the hypotheses we originally proposed. 
One such would involve the specific identification, within each school, 
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of users and non-users of the college files^ combined with a list of the 
colleges users obtained from the GIS files and the colleges each group 
applied to* An alternative approach would involve matching a GAG school 
with a non-CAG one^ or comparing CAG schools with other high schools that 
use different ways of providing students with college Information* 



Reported Effects of CAG on College and Career Plans 

The anecdotal evidence suggests that GIS helped some students make 
more appropriate college decisions and choices^ and will give the reader 
a feeling for the kinds of changes or results CAG was observed to produce 
A question of interest to us was whether effects of CAG would be evident 
and docymentable while students were still in high school* Could we 
determine whether students would make any changes in their current high 
school program 1 as a result of information they received from the compute 
or in their plans for after high echool graduation? The reactions in tht 
section Include effects on both occupational»-career--and college plans. 

Three liaisons (from Schools A^ and E) noted student program 
changes^ which they attributed to use of the CAG system. Their reports 
commented on students who began to realize that they should have better 
high school preparation in their subject of Interestsor in a subiect 
related to a college majo^ or to a job they were considering* One stu- 
dent decided she should take more typing in preparation for an office 
job* another student interested in a college secretarial major elected 
high school courses in clerical procedures and speedwrlting after reading 
the computer's Job description. The liaisons reported that students 
interested in nursing careers realized they needed more high school 
chemistry and math for nursing schools; in School because of their 
Interest in nursing^ some students discovered that their school offered 
a course in psyehobiology. 

In terms of helping students make plans for college^ one of CAG's 
values cited by the liaisons was that it often suggested new options that 
students had not considered; some examples follow* A student interested 
in ceramics and wanting to attend SUNY discovered that Alfred University^ 
a SUNY college^ met his requiranents* Students interested in medical 
technology found programs available at many colleges not prevtously con- 
sidered* A student interested in oceanography located a school in 
Pennsylvania through GIS; he previously thought he would have to travel 
as far as Florida, A student interested in criminal justice applied to 
CUNY only for John Jay College^ until he discovered that a college in 
Massachusetts offered such a program. And at least tw students returned 
to School E to tell the liaison how happy they were at (two different) 
two-year colleges In New England^ information about which they had '-first 



The original intention was to note all "significant" changes in high 
school prograns in one or two of the pilot schools^ and to determine if 
users requested more such changes than non-users. This approach was not 
feasible^ so we asked liaisons and other staff about changes in high 
school and post-high school plans. 
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learned from the computer," The School A lialion reported that one stu- 
dent had applied to two two-year colleges that appeared on his printout^ 
was accepted at both^ and will go to one to study agronomy* Another 
foreign student without a residence visa wrote (with help) to every 
college that offered engineering in every state; he found several that 
would accept him and give him financial aid. Another student who felt 
aha had to go to CUNY because of costs^ found Fordham University and 
Boston University — both of which accepted her with full (ffilOP) scholar- 
ships. 

Other illustrations indicate that CAG often suggested new ideas 
about occupations^ so that students could better relate their current 
interests and future educational plans to occupations * A student found 
that the field of urban planning would let him combine his strengths in 
social studies and art* Another student^ who worked as a CAG monitor and 
helped to batch requests^ indicated a strong desire to learn more about 
computers and become a progr^mer*^ A student who wanted to work with 
mentally retarded individuals learned that she didn't have to be a teacher 
but could accomplish her desire in other settings. And another found that 
the occupation of medical secretary would let him combine his interests 
in medical technologies and secretarial work, A student interested in 
advanced math found at least 20 different occupations which could use 
this strength* A liaison happily reported to us that art majors began 
to realize that they could find iobs in many areas other than fashion 
design. 

Still other affects were related to students^ becoming more realistic 
in their future planning* One student^ of limited acadanic ability^ was 
Interested in being a pediatrician but had no idea of the amount of 
schooling involved; she discovered that she could still '-work with 
children'* as a licensed practical nurse. Another student decided not 
to go into the family business because it became increasingly clear to 
her that she could not best use her interests j which were important to 
her to pursue^ in that setting* In several instanceSj especially in the 
health field^ students discovered alternatives (e.g*^ to becoming an M.D,) 
that were much more in line with both their abilities and their wish not 
to pursue many additional years of education. Others decided to go on to 
college after having viewed the range of jobs available with only a high 
school diploma* 

The responses of students at the terminal and afterwards in follow- 
up sessions^ were uniformly favorable, A few students were skeptical. 
The largest nirober of professed skeptics seOTed to be at School wher6 
several students made comments to IRDOE staff about the "dimb" or "useless" 
computer--i*e*^ the colleges and/or careers listed were^ apparently^ 
beneath consideration. It is difficult to determine how negative these 
students were toward CAG^ since the majority were recruited by word-of- 
mouth from seCTiingly satisfied either student users. In all schools^ it 
was not uncoOTiion to have students drop in and ask if they could "use the 
computer^" and there was evidence that one user's excltOTent generated 
interest raong other students, 

i 

The CAG project has produced some "computer buffs"; several students ex- 
hibited as much sophistication about the intricacies of the GIS progran 
as any adult user. 
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Their interest^ often quite obvious in their facial expresslonSj 
ranged from the technological aspects ("You mean this really comes from 
New Hampshire?" and "This machine Is fantastic 1") to spontaneous asser- 
tions of the value of the information received ("Everything you wanted 
to know")* One student said^ "All schools should have this^" and another 
insisted that her sister (at another school) be allowed to come in and 
use it. Only one instance of resistance to the process was reported: One 
student (at School A) found the experience of having to make choices so 
threatening that she became "angry with the computer^" called it "stupid^" 
and left the terminal area "close to tears." 

We have already mentioned the behavior of students in "testing" 
their decisions; they put characteristics into the computer^ and cane 
away with a list of colleges or occupations that included the specific 
one they had already decided on* In our early vlewj this behavior did 
not sean productive; later ^ however^ we cane to realize that it helped 
students to make explicit an often internalized^ nonverbal decision- 
making process* All students wanted to take their copy of the prlntoutj 
and were observed studying their own and examining those of their friends. 
The fomat of the GIS printout (o£ a search) makes it readily under- 
standable that each choice makes a difference^ and that decisions have 
consequences. Indeed^ as we shall see In the next chapter^ this 
decision-making experience seamed to make GAG users (statistically) better 
at the skill than non-users. 



93 



ERIC 



CHAPTER VIII ^ 



OUTCOMES: IMPACT OF CAG ON STUDEEfT USERS 



/ By the end of Period 1 (June 1974)^ the ^tent of use of CIS as well 
as the student and staff interest and enthusiasm evoked by this project 
made clear the feasibility of computers-assisted guidance in the high 
schools. Apart from feasibility^ the remaining question Is whether the 
effects on students are sufficient to justify the continued or expanded 
use of CAG. The results of the impact tests to be discussed in this 
chapter present quantitative evidence of these effects. The tests were 
given in Period 3^ early in 1975. (See Figure 4^ Chapter III for 
details of the administration*) 

Hypotheses concerning CAG effects were examined with an attitudinal 
measure and several cognitive measures. The first hypothesis was that 
student uaers^ as compared with non-users^ would prefer the computer as a 
valuable source of college and career Information. Secondly^ users would 
have more knowledge of terminology and concepts that are used to talk 
about colleges and occupations. Thirdly^ users would show more ability 
to describe and organize Information about colleges and careers in ways 
that reflect important parmeters. The final^ and perhaps most Important^ 
hypothesis was that users would demonstrate better knowledge or under- 
standing of the narrowing concept underlying the declson-making logic 
which G IS CTiploys. 



Limitations Concerning the Test Results 

Before presenting the outcomes^ several assimiptlons^ limitations^ 
and definitions will be clarified* First^ this project la of necessity 
concerned with the impact of TSC's Guidance Information Systan on stu- 
dents; therefDre^ many of the specific test Items and the scoring categories 
developed reflect both the strengths and the weaknesses of that system. We 
did make every effort to select test items having general^ widespread im- 
portance in conceptualising college and career Infomatlon^ as well as 
having relevance to the specific hypotheses. We can assume that stoilar 
findings would apply to an alternative guidance system^ provided of course 
that it is more or less comparable to TSC's (e,g,^ employs the same 
decision-making logic); and that a parallel vocabulary test, using terms 
from the alternative systCTi's particular language^ would yield similar 
results. 

The definition of "use" Is an Important consideration. Of the student 
users tested, over half had never had direct interaction, 1-e*, were not 
present at the processing of their request, and some of them may never 
have seen the terminal. For any individual student, CAG use may have 
differed with respect to many variables: 
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Whether or not the studant reviewed the Student Study 
Guide (or a facsimile)^ and whether the review in- 
volved whole-clasSj small-group^ or Individual orien- 
tation sessions* 

The particular file (college or occupational) the 
student requested infortnation about. 

Whether or not the student requested a file search^ 
a description of a specific college or career^ or 
both a search and a description. 

Whether or not the student was present during the 
processing of his/her request. 

Whether or not the student received a copy of the 
printout. 

Whether or not the student engaged in followup ac- 
tivities^ including discussion of the output with 
the liaison or counselor^ writing for infomation^ 
going to a catalog^ etc. 

For the study of CAG tapact^ we did not attOTpt to Isolate which 
CAG-use variable(s) was critical. That is^ in delineating user/non-user 
groups^ we did not attempt a more refined breakdown of^ for eKample^ 
college/occupational file users or direct /batched users. About a third 
of the users tested had had experience only with the occupational file^ 
and some portion of users had exposure only to college files; many users 
had received only descriptions of specific colleges or careers^ while the 
majority had done searches* The effects of these factors^ If any^ 
would tend to reduce user/non-user (U/NIJ) differences. Although some 
suppositions are possible fTom our evidence^ future clarification with 
respect to the effects of these variables (singly and in combination) 
on U/NU differences would be highly desirable. 

The other side of this question is the definition of non-use. It 
Is very likely that many students designated here as non-users may have 
had some exposure to GAG In whole-class orientation (or followup) sessions^ 
which could have involved the Student Study Guide (or an examination of 
printouts). If this factor played a part^ however^ its effect would be 
to again reduce U/NU differences. 

The recency of the CAG use prior to the impact testing may also 
have had an Influence. For nearly three-fifths of the users tested^ the 
CAG experience preceded the testing by a half to a full yea^. As will 
be seen^ the very encouraging results give evidence for long-range 
carryover effects of what Is a relatively brief experience In a high school 
student's life. 

The m^Bt crucial question affecting interpretation of the test re- 
sults Is whether there were initial differences between users and non- 
users. It is always difficult to conduct carefully controlled studies 
in classroom situations* We could not assign students randomly and 
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equally to the user and non-user groups^ and school policies at various 
tlmea encouraged use by higher- or lower-ability students. We have al- 
ready discussed possible ability differences bet*/w^een the users and non- 
users in the section on Evaluation Methods^ as well as our reasons for 
assuming that any euch differences which may exist cannot account for 
the obtained results ^ 

The statistical analytic procedures Qvercame some of the problems^ 
but questions remain as to whether school variations in implsnentation^ 
use of GIS^ and so on^ affected the outcomes; and as to whether there 
were other in^ortant ways in which users differed from non-users^ es- 
pecially with respect to motivation. Did users tend to be volunteers? 
According to the liaisons' reports^ the proportion of self -selection 
differed from school* to-^school and from period-to--perlod^ and ranged from 
little to naich. At School A volunteering was almost nonexistent in 
Period 1^ and students had to be "dragged in," At School too^ there 
were few volunteers* Schools B and D did considerably less active 
recruiting of students than A and with more reliance on self-selection. 
At School the users were primarily volunteers^-students interested or 
curious enough to "drop in." The school-to-school differences in self- 
selection^ however^ do not explain school-to-school variations in results* 

The results^ to be presented next^ demonstrate overwheMlng benefits 
to students with CAG experience. Whatever the possible initial differences 
betoeen users and non-users^ we conclude that they do not negate these 
benefits. The findings will be discussed In the following order: 

1. Preferred Sources of College and Career Information; 

2. Vocabulary; 3* Self College Questionnaire; 4* Ideal 
College Questionnaire; 5* Self Occupation Questionnaire; 
6. Ideal Occupation Questionnaire; and 7. Decision-Making. 

Vocabulary and Decision-Making were given In all schools; the college 
and occupation instruments (Nos« 3^ 4^ 5^ and 6) were each given in 
selected schools (Bj D^ and E)* The students tested were mainly seniors^ 
except for School E where 84% were juniors* No one student took more than 
two of the tests, (For details^ see Appendix^ Table A2#) 



Preferred Sources of College and Career Information 

The two attitudinal questions listed several sources of college and 
career Information^ which were similar although not Identical » From each 
list the student was to choose the three (s)he considered most valuable. 
Choices of the computer as a valuable source were of special Interest. 
Table 12 on the following page shows^ by school^ the proportions selecting 
each source as one of the three most valuable; for each question^ the 
sources are listed in overall rank order of preference. 

As a general conment on this table^ U/NU differences were usually 
riot largej and there was a striking degree of similarity mong the sep- 
arate schools as to the rank order of student preferences. Also^ the 
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TABLE 12 

USERS' AND TON-USERS' OPINIONS OF Tlffi THREE MOST VALUABLE 
SOURCES OP COLLEGE AND CAREER INFORMATION, BY SCHOOL 
(Resulta in Percentages)'' 





School ^ 


All 




Sourcas of: 


A 


B 


C 


D 


E 


Schools 






U ; 


NU 


U 


NU 


U 


NU 


U 


NU 


U 


NU 


U 


NU 


U+NU 


College Infomation 




























College Office 
or Counselor 


79% 


97% 


76% 


87% 


76% 


82% 


78% 


84% 


86% 


947 


79% 


88% 


84% 


Visits to College 


47 


71 


68 


82 


60 


76 


70 


52 


60 


64 


61 


71 


66 


College Catalogs 


51 


47 


51 


58 


55 


51 


46 


52 


52 


42 


51 


50 


51 


Friends at College 


28 


38 


39 


36 


58 


36 


38 


25 


20 


28 


35 


33 




Computer 


40 


23 


37 


22 


12 


20 


19 


35 


62 


47 


36 


28 


32 


Love joy' s or 
Barron* s 


40 


15 


24 


9 


24 


20 


30 


32 


16 


19 


27 


18 


22 


Parentis or 
' Guardians 


7 


3 


0 


2 


9 


7 


14 


10 


4 


0 


6 


5 


5 


School Library 


4 


3 


5 


4 


0 


6 


5 


10 


0 


3 


3 


5 


4 




4 


3 


0 


0 


6 


2 


0 


0 


0 


J 


o 


L 


2 


Career Information 




























Career Office 
or Counselor 


O D/g 


^ J. /g 




877o 


78% 


95% 


81% 


87% 


82% 


957 


82% 


91% 


86% 


Talking with 
People in Dif ^ 
ferent Careers 


86 


79 


88 


73 


91 


93 


78 


77 


72 


69 


82 


80 


81 


Visits to Places 
of Work 


42 


53 


56 


53 


53 


56 


70 


61 


60 


56 


56 


56 


56 


nnrmDll teT" 


28 


32 


34 


25 


16 


13 


14 


26 


52 


36 


31 


25 


28 


Dictionary of Oc- 
cupational Titles 


40 


27 


29 


33 


22 


22 


16 


26 


26 


33 


27 


28 


28 


Parents or 
Guardians 


2 


12 


7 


13 


25 


14 


27 


13 


4 


11 


12 


13 


12 


School Library 


14 


6 


3 


11 


13 


2 


14 


10 


2 


0 


8 


5 


7 


High School Friends 
(NO 


2 0 
(43) (34) 


3 5 
(41) (45) 


2 5 
(33#)(55) 


0 0 

(37)(31) 


2 

(50) 


0 

(36) 


2 2 
p04f^(201) 


2 



^These two questions (given in Period 3^ January or February 1975) were placed 

at the end of the Decicion-Making test. Responses were analyzed only for tliose 
who gave 3 choices* 

*N for career sources is 1 less than the number shown. 
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(overall) rank order of the college sources was vary similar to that for 
the career apurces. Note that the college and career percentages cannot 
be directly compared; since there were nine college sources but only 
eight career sources listed as opinions, and since each column sum must 
be 300%, the career percentages are necessarily each higher than any 
corresponding college percentage*^ 

College Information^ Considering first the preferences for the 
various callega sources^ in every school the students (U+NU) gave the 
top tank to the college office or counselor. Next in rank order were 
visits to colleges, and then college catalogs. The computer ranked in 
the middle* Almost no students regarded parents, the school library, or 
their high schoDl friends as valuable information sources. The only 
notable exception to the agreement anwng schools in rank-order pattern 
was School E, where students ranked the computer higher than did students 
in any other school. In contrast to School E are the students at School 
C who valued the computer least. 

In fact, the proportion of School E non-users (47%) selecting the 
computer as a valuable source was larger than the proportion of any other 
school -s users selecting this Dption. This unique feature of School E 
students' responses might be attributable to the fact that n^st of them 
were iuniors . whereas in the other schools only seniors were tested. An 
alternative consideration is the very small sl^e of School E which could 
serve to increase the conmunlcatlon between users and non-users; also 
to be considered is the fact that School E had the largest (eKcept for 
Period 3) proportion of descriptive uses of any other school. 

Comparing users an^ non-users, for all schools somewhat more of the 
users preferred the computer (U, 36%' NQ, 28%) and Lovejoy's or Barron's 
(U, 27%; NU, 18%); at Schools C and D fewer users than non-users preferred 
the computer. For all schools combined, somewhat more of the non-users 
chose the college office-counselor (U, 79%; W, 88%) and visits to colleges 
(U, 617o| NU, 71%) p Students may not have differentiated precisely between 
'^counselor'* and "computer." To many student users, "computer" may have 
included "counselor" (i.e., liaison), and we cannot be certain what they 
meant. For Schools A, B, and E where the greatest proportion of users 
regarded the computer as a valuable source of information, there was also 
the greatest difference bett^een users and non-users In their preference 
for college office-counselors. Schools C and D, which have the more 
elaborate college advistng office, did not follow the Bmm pattern- At 
Schools C and D, not only was the computer less valuable to users than 
non-users, but there was greater similarity between U/NU groups in rating 
the college office* 

Car e e r I n f o rma t lo n . Looking at preferences among the eight listed 
sources o£ career Infomation (bottom half of Table 12), there are obvious 
similarities to the college source preferences. Here the career office or 
counselor ranked highest, and, as with the college choices, especially so 

^Taking 8/9 of the career percentage (or multiplying by ,9) gives an 
approxmation of the career percentage if there had been 9 career choice£i, 
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for non-users (U^ 82%; NU^ 91%)- Nearly as high^ and in some InstanCQS 
higher^ was talking with people in different careers. Visits to places of 
work^ ranking third (U-WU)^ was generally not valued nearly as highly as 
were visits to colleges* The computer again ranked in the middle^ just 
barely above the Dictionary of Occupational Titles * And again j students 
gave parents^ the school library^ and their high school friends a low value • 

Aa with the college sources^ non-users in each school valued the 
career offica-counselDr more highly than users^ while more users chose the 
computer as one of 3 sources than did non-users (estimated overall per- 
cantages^ 28%; NU, 21%)^ Both groups of students gave the computer a 

lower value for career infonsiatlon than for college infofmatlon* The fact 
that users viewed the computer--and there "computer" is identical to the 
TSC syatein-^^as less valuable for career than for college information 
accords with our experience and liaisons' reports^ which Indicates that 
the CIS occupational file is more poorly organised and difficult to use 
and interpret. r 

In the separate schools^ again the students in School E (both users 
and non-users) ; ranked the computer option more highly than did students 
In the other schools; while students In School C and the users in School 
D ranked the computer very low as a career Information choice* In 
Schools A and users ranked the computer less highly than did the non- 
user groups. In comparing the proportions of users who valued the library 
as an informational source^ In almost every school the library was more 
highly valued for career information than for college Information. Since 
use of the GIS occupational file is presented as a "first step^" and stu- 
dents were encouraged to follow up with other library sources^ this is 
not an unexpected result^ and one which reflects use in the schools ^ 
Furthermore^ the fact that^ with the exception of School the schools 
did not have a career progrm and yet the "career office or counselor" was 
so highly rated by the users^ lends additional support to our supposition 
that students did not differentiate between "counselor" and a computer-^ 
assisted counseling experience* 

Other studies have consistently reported that "students like to use 
the computer^" but few of thffli asked the students to make a choice* Thus^ 
our relative rankings of preferred sources are not comparable to the 
results of previous studies. In the present situation^ a third to a fourth 
of the students chose the computer as a valuable choice. We know from 
talking to students^ from the number of return uses^ and from staff reports 
that students In the pilot high schools also liked using the computer. 

In sunmary^ when all schools were combined^ there was some support 
for the hypothesis that users^ as compared with non-users^ would rank the 
computer as a valuable source of college and career information. But this 
hypothesis did not hold true in all individual schools^ and neither users 



In 1975 when these questions were administered^ only School B had a 
separate^ active involvement in career education; little career service 
was available at Schools A and the career counselor at School C was 
engaged in general counseling duties, and at School E the sane person 
fulfilled both college and career advising responsibilities. 
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nor non-users ranked the computer as thair first choice; instead, they 
gave It an intermediate value ati^ng the listed options. For the college 
Eource questions^ non-users In Schools C and D gave the computer more 
favorable rankings than did users. The fact that thesa two schools are 
the most acad^tc-oriented and have the most elaborate college advising 
offices suggests that students in Schools C and D may have^ In reality^ 
sources better than the computer a\^ailable to them; and those who have 
already used CAG \^ere better able to make such a judgment than non=users* 

The relative ranking of the cnmputer apparently has little relation 
to what a user ''gets out*' of the CAG experience. We could not detect any 
relationship between these U/NU attitudes and the U/NU differences on the 
cognitive impact maasures. Thus^ for exMple^ although School E users 
were the most enthusiastic about the computer^ the results will show that 
on the Decision-Making test they differed least from non-users. But the 
School C users^ who were not In general ardent computer supporters^ did 
demonstrate superiority over non-users on both the Decision-Making and 
Vocabulary measures* Further Investigation Is needed to determine to 
what relative preference for the computer as an infortnational source 
relates . 



Vocabulary 

To ascartaln whether GAG users had a greater knowledge than non-users 
of the terms and concepts used to describe colleges and careers^ we 
developed a 12-item multiple-choice Vocabulary test. The six college and 
six career items were selected from the Student Study Guide as represen- 
tative not only of the terns used In GIS. but of Ideas we felt were 
important and in coimnon descriptive use.l 

Table 13 shows the results* Looking first at the "Mean Advantage to 
Users" column^ note that users In School which has the lowest academic 



TABIJE 13 



MAN VOCA 


BULARY SMRES 


FOR CAG 


USERS 


AND H 


)N-USERS, BY SCHOOL* 


School 


U+NU 


Users 


Non 


-Users 


Mean Advantage 


N 


Mean 


N 


Mean 


N 


Mean 


to Users 


A 


75 


5.9 


48 


6.4 


27 


4.9 


+1.5 


B 


79 


6.6 


35 


6.5 


44 


6.6 


-0.1 


C 


92 


7,5 


33 


8.2 


59 


7.0 


+1.2 


D 


86 


8,2 


43 


8.4 


43 


8.1 


+0.3 


E 


81 


7.6 


55 


7.7 


26 


7.2 _ 


+0.5 


All Schools 


413 


7.2 


214 


7.4 


199 


6.9 


+0.5 



i^Maxi™m possible score - 12# 



The Vocabulary it^s consisted of: employment outlook| university; ac- 
creditation; occupational cluster; elitry-level salary; structural work; 
coeducation; level of formal education; manual dexterity/spatial relations; 
total enrollment; annual tultionj fees^ room and board; and private vs. 
public colleges. 
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level of the five schools^ beneftted most; School C users beneftted 
nearly as much^ and School B least. However^ the magnitude of the dif- 
ference between users and non-users, statistically speaking^ must be 
regarded as the same in all schools (use X schools interaction^ not sig- 
nificant). Thus,, variations among schools in students and In method and 
process relating to CAG gave no jl^nif tcmijly grea advantage to users 
in one school than to those In any other. 

Next we can examine the question of prljnary interest--the difference 
between users and non-users. The overall mean for users was 7,4 and that 
for non-users was 6. 9^ this differencse is significant at the =005 level. 
Thus^ we can conclude that users per formed significantly better than 
non*-usera on the Vocabulary test. 

Looking next at the overall differences in level □£ perfdrmance among 
the five schools (see U+NU colmnn In Table 13)^ the mean differences were 
even more clearcut (p <^001), This result Indicates that^ grouping users 
and non^users together^ average performance of students on the Vocabulary 
measure was significantly better at some schools than at others. The rank 
order was: School D (highest)^ then Schools E and School and 
School A- The five overall school means correspond approximately with our 
information about the academic ability levels of the student populations 
in these schools. This outcome is expected^ since any vocabulary measure 
also taps ability* 

In suranary^ CAG users clearly and unequivocally did better than non- 
users on the Vocabulary testj demonstrating a greater knowledge of tmpor- 
tant GIS terms and concepts. Furthermore^ largely because any measure of 
vocabulary is closely correlated with ability^ the schools differed In 
overall performance level with means distributed roughly in accord with 
performance on citywlde achievCTient tests. 1 Moreover^ although the over- 
all performance of some schools was better than others^ this in itself 
did not give a greater advantage to users in those schools. 



Description and Organization of College and Career Inform ation 

To determine whether CAG had an effect on the way students deacribe 
and organise infomation about colleges and careers^ we developed two 
parallel open-ended questionnaires for each of these areas^ with space 
for writing 10 responses* Scoring methodologies and analysis procedures 
were similar for the four questionnaires,^ The objective was to compare 



That Is^ roughly in accord with the proportion reading at or above 
grade level as measured by citywlde reading achievement tests, 
2 

Two basic scores were developed --number of statements and number of 
categories. As indicated in Chapter III^ a statement was defined as 
containing one piece of Information; two sentences by the same respondent 
containing identical information was scored as one statement. Categories 
were generated by grouping similar statements together. Wherever pDssible^ 
GIS category classifications were used. See pages 23-34* 
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users and non-users ^ith respect to the amount^ variety^ and kind of 
Information generated. No responses were scored for accuracy: our aim 
was to ascartain the factors atudents considered important when answering 
the questions. 

The Self College ^estionnaire asked students to list '-up to 10*' 
important characteristics of the college they would most like to attend^ 
as well as the name of the college^ The Ideal College Questionnaire asked 
the respondent to list questions that an "ideal college advisor*' would 
ask a student seeking college inforiMtion, The Self Occupation Question- 
naire asked the respondent to list the one job or occupation (s)he was 
TDOst interested in having after completing all schooling^ and to describe 
'*up to 10*' Important things about it. In the Ideal Occupation Question- 
naire^ the student was asked to pretend that (s)he was an ''tdeal guidance 
counselor^'* and to indicate questtons that would help a particular senior 
(Russ)j decide on an occupation. No class or student took more than one 
of these jEour instruments; they were not administered in Schools A or C* 

The results are presented in Table 14* It is interesting to note^ 
first of all^ that the college and occupation "ideal** questionnaires^, for 
both users and non-users^ generated more statements and more categories 
than the respective *'self** questionnaires. In fact^ we found that the 
"ideal" Instruments were easier to score and interpret^ with fewer am- 
biguous responses. It may be that the *'ideal** instruments^ by removing 
the situation from the self^ make It less likely that the student's own 
personal concerns will interfere with responding. Another general com- 
ment Is that both college questionnaires, in contrast to the two occu- 
pation instrmaents, yielded fewer statements that we could not score or 
classify*! One possibility iS' that these students have a clearer 
understanding of the information necessary for decisions about college 
than they do for deciding about occupations; or perhaps occupational 
decisions are inherently more complex. 

College Information . As the top part of Table 14 Cp# 91) shows, the 
differences between users and non-users were quite similar on the Self 
College and the Ideal College Questionnaires, except that differences 
were more pronounced on the latter* On the average, users made more 
statements than non-users (Self, 6*3 vs* 4*7; Ideal, 8.8 vs, 6*9), and 
gap loved IPO re categories (Self, 5.9 vs* 4.5; Ideal, 8*2 vs. 6.3), This 
difference resided al^st entirely in the users' greater employment of 
GIS-related categories; mean U/NU differences for non-GIS-related cate- 
gories were very small or zero. Thus, It is clear that the users 
generated more information and a greater variety of Information that was 
related to the GIS classifications. 

A further analysis of the categories produced showed that on the 
Self College Questionnaire, for all respondents, there were 41 discrete 
categories^ Users generated 38 of these, 11 of ^hich were unique to them; 
non-users generated 30 categories, only 3 of which were exclusive to them. 
On the **idear* instrraient, the user plus the non-user groups produced 



On the Self College Questionnaire, for exmple, only 9 of the 268 state- 
ments were ambiguous or too vague to classify* 
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TABLE 14 

^ffiAN SCORES ON COLLEGE AND CAREER INFORMATION QUESTIONNAIRES^ 

FOR USERS AND OTN-USERS* 



College 
Questionnairs 


Sell: 




Id 


eal 


Users 


Non- 
Users 
(N=28) 


Mean Advan- 
tage to Users 


Users 
(N-79) 


Non- 
Users 
(N=83) 


Mean Advan- 
tage to Users 


N Statements 


6.3 


4.7 


+1.6 


8.8 


6.9 


+1.9 


N Categories 


3.9 


4.5 


+1.4 


8.2 


6.3 


+1.9 


CIS Categories 


4.0 


2.6 


+1.4 


5.9 


4.3 


+1.6 


Non-GIS Categories 


1.9 


1.9 


0 


2.3 


2.0 


40.3 


Occupational 


Self 




Id 


aal 


Users 
CN=69) 


Non- 
Users 
(N=44) 


Mean Advan- 
tage to Users 


Users 
(N=63) 


Non- 
Users 
(N''54) 


Mean Advan- 
tage to Users 


N Statanents 


6.5 


5.7 


+0.8 


9.0 


8.0 


+1.0 


N Categories 


5.7 


4.0 


+1.7 


8.0 


6.8 


+1.2 


CIS Categories** 


3,6 


2.4 


+1.1 


3.1 


2.7 


+0.4 


Non-GIS Categories 


1.3 


1.0 


+0.3 


4.2 


3.6 


•K).6 


Irrelevant or fm~ 
biguous Categories 


0.8 


0.6 


+0.2 


U . / 




-rU . d 


Job-Ortented Categ, 


4.0 


2.8 


+1.2 


2,2 


1.9 


+0.3 


Personal-Oriented 
Categories 


0.9 


0.6 


+0.3 


4.4 


3.5 


+0,9 


Neutral Categories 
("Consider/' 
"Consult") 


0.0 


0,0 


0 


0.7 


0.9 


-0.2 


Irrelevant or M,- 
biguous Categories 


0.8 


0.6 


+0.2 


0.7 


0,5 


+0.2 



*Self College was given in School D; Ideal College In Schools D^ and E; 
Self Occupation and Ideal Occupation in Schools B and E# No class or 
student took more than one of the foui'. For discussion of scoring^ see 
text. 



**Small portions of these means^ nearly equal for users and non-users^ 
represented GIS college rather than CIS occupational categories; mean GIB 
college categories^ Self^ U 0,4^ NU 0*3; Ideal^ U 0*8^ NU 0*6, 



statCTients that were classified In 55 distinct categories (U^ 47; NU^ 42); 
8 of these categories were unique to the users- groups and again there were 
only 3 categories generated exclusively by non-users. 
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In terms of the statCTents made by respondents (U+WJ)j 63% and 62% 
of the BtatCTients made on the Self and Ideal College Questionnaires^ 
respactively^ were judged as GIS-related. That is, about two-thirds of 
what students listed on their questionnaires as important information 
about colleges is available to them in the GIS files. Both users and 
non^^users tended to consider similar kinds of information important^ but 
with somewhat differing emphases. The categories most frequmtly listed 
by both groups included the college- s competitiveness, the fact that it 
offered a specific major, its tuition^ locatiort^ and whether it was a 
twO'^year or a four-year institution (all in GIS); and its reputation. 
Where there were U/NU differences in this respect, on the "self" instru- 
ment users much n^re freqt»ntly described the wide range of majors 
available, in*-college characteristics such as small classes or few lec- 
tures^ the social climate^ and special tests for admission* Non-users 
more frequently described transportation problems, tuition^ and the 
physical plant. On the ''ideal" instrinnent, the five most frequently 
mentioned categories (all in GIS) were: major field of study, competi- 
tiveness, tuition, location, and two-year vs* four-year Institutions* 
More users than non-users made statements about competitiveness, location, 
total enrollment^ and coeducation (all in GIS); and about "living away 
from home," 



Results from the two open-ended college questionnaires were very 
similar* It is evident that GAG users have available more college-related 
information than non-*users, and organize that information in terms of a 
wider variety of categories* The users' advantage can be attributed to 
their experience with GAG, since tmst of the U/NU difference lies in 
information available in GIS^ information that tends to appear as unique 
user responses. 

Although about two-thirds of the responses of both users and non- 
users Involved information available In the TSC system, a little over a 
third of their Interests are not reflected in the GIS files, Exanples 
of these concerns include prerequisites for college majors^ size and 
reputation of the college department, size of classes, lectures vs. 
seminars, and required and elective courses. In our opinion many of 
these concerns have legitimacy and lend support for systCTi modifications 
more responsive to the New York City high school student. 

Career Information * While the responses to the two college ques- 
tionnaires were very similar in content, this was not as true of the two 
occupation Instriments, which ^e^ed to tap somewhat different student 
concerns* For this reason, the findings from each version will be dis- 
cussed separately* 

As Table 14 shows, on the Self Occupation Questionnaire the mean 
difference in nimber of statOTients and niunber of categories favored the 
users, and particularly so in terms of the variety of Information 
generated (i.e., nraiber of categories). As with the college instruments, 
a primary reason for this user advantage was the greater number of GIS- 
related categories they employed. Analyzing these GlS-related responses 
further, we found that (just as with the college questionnaires) about 

104 



93 



two-thirds of all statOTients by both users and non-users reflected in- 
formation In the TSC system (U^ 65%; NU^ 66%)* These data strongly^sug- 
gest that both groups are similar in what tuey think is important about 
occupations^ and that GIS contains much information of general Importance 
to students. 

For all respondents combined^ there were 92 distinct categories; 
users generated 85 categories and non-users produced only 54. There was 
also a striking difference in the ninnber of unique categories (U^ 38; NU^ 
7); more than half the 38 categories unique to the user group were GIS- 
related^ but none of the 7 categories unique to the users were related to 
the TSC syston. 

In an attraipt to identlfyfemore specifically the u/NU differences In 
kinds of categories produced^ we next re-grouped all the statements Into 
job-oriented and personal-oriented categories. Here the grouping depended 
on the student's wordlng--that is^ whether the response referred directly 
to Job characteristics (e.g^^ '*Thls Job requires heavy physical work") or 
whether it was personally oriented (e.g*^ ''You must be strong"). Even 
though the content might appear similar, these were defined as different 
categories . 

Job-oriented responses characterized about three-fourths of the 
statements of both groups (u;^ 71%; NU^ 77%). But this does not mean the 
groups were alike in this respect. We see from Table 14 that the user 
group generated a greater nimber of Job-related categories than did non- 
users (means - 4.0; NU^ 2.8) -^indicating more varied responses* For 
both groups^ personal-oriented categories and irrelevant or anblguous 
categories constituted only small portions of the total^ and there were 
almost no "neutral" responses. 

The final analysis of the Self Occupation Questionnaire (and not 
Included In Table 14) concerned responses that were schooling- or training- 
related^ 1 since 39% of the user respondent group (from Schools B and E 
combined) had experience only with GIS college data files. About two-thirds 
of the users (62%) made statements on the Self Occupation Questionnaire 
concerning the amount or kind of training needed for their chosen occupa- 
tloDp This Is in sharp contrasts to the one-third of the non-users (34%) 
who discussed the training or educational requirements for an occupation* 
(Similarly^ users averaged more training categories than non-users--1.5 
ajid 0.9^ respectively.) 

It Is interesting to note ttat of the 113 students tested^ only 2 
did not even Indicate an occupation they might llke^ and both were non- 
users. This recalls an analysis done of the 11th grade respondents at 
School D on a question on the Initial Survey asking th&m to list first^ 
second^ and third occupational clioices. Although there was no appreciable 
difference between CAG users and non-^users in terms of the nmnbers who 
listed one^ two^ or three choices^ there was a difference in the percentages 

^The schooling- or training-related categories include mention of a 
collsga or other place for training^ years of education^ or training 
needs for the occupation^ kind of degree required^ etc. 
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not responding to the question, ^ong the non-'Users^ 7% gave no response^ 
whereas every respondent who had used CAG listed at least one choice^ if 
not TOore, A chi**square test of this difference was significant at the 
• 02 level of confldenca. 

For the Ideal Occupation Questionnaire^ Table 14 shows that the re- 
sults were in many ways similar to those for the Self Occupation Question- 
naire^ and that the mean differences In the various scores again clearly 
favored the users. The "ideal" version^ towever^ while generating more 
statements and categories, tended to yield fewer GlS-related categories 
and many mo re non-GIS-related categories than the "self" instrument; it 
also produced fewer Job-oriented and many more personal-oriented categories. 

Although users produced more and more varied responses than non-users^ 
looking at the GIS/non-GIS breakdown In Table 14 shows that the mean u/Mj" 
differences were not large. Users employed on the average 3-1 GIS cate- 
gories^ and non-"Users 2.7- Both groups generated more _non ^GIS-related 
categories (means - 4.2; NU, 3.6), with a si ghtly greater advantage 
to users than non-users on this score. Thus, half the overall U/NU mean 
difference In total nmnber of categories reflects information not avail- 
able in the TSC systra. Why the "ideal" version generated more (and more 
varied) non-GlS Information than the "self" occupation version or either 
of, the two college questionnaires is not understood. It could be that 
students' conceptions of giving career counseling were less well defined, 
possibly because of their relative Inexperience with career counselors. 

In exOTlning the Impersonal-personal breakdown, we see that the 
"Ideal" version elicited many more personal-^orlented statements (means: 
4.4; NtJ, 3*5) than Job-oriented ones (meansi U, 2*2; NU, 1*9); and 
this is especially true for users. This finding is just the reverse of 
that for the "self" version* We suggest that the instructions make It 
more appropriate for a respondent to answer the "self" question in Job- 
oriented terms, while for the "ideal" Instriment it seems more appropriate 
to foitaulate questions about "Russ" in personal-oriented terms. Since 
these effects were more marked for users. It might be that they were mave 
attentive to the specific wording of the question*^ 

Of the total of ^8 distinct categories, GAG users generated slightly 
more than non-*users (U, 70; NU, 64), but this difference was much less 
marked than on the Self Occupation Questionnaire (where users' responses 
were classified In 85 categories, and non-users' In 54)* In terms of 
the proportion of responses related to CIS, users and non-users did not 
differ much^-the mean advantage to users was 0*4, as compared with 1*1 on 
th^'self" Instrmnent. This finding and the analysis of Job- vs* personal- 
oriented statements support our assumption that the two versions tap 
different studant concerns. 



Since using GIS (or other similar systOTs) requires the user to attend 
to the * Wording' of his/her request, there might be sOTe carryover effects* 
Also to be considered^ however, are Initial differences in this respect 
between users and non-users. Although we can present little additional 
evidence in support of either explanation, it suggests Important future 
lines of Inquiry* 

106 



95 



Some final words about the content of the responses to the Ideal 
Occupation Questionnaire. A smaller proportion of users than non-users 
(U^ 11%; NU, 27%) suggested specific occupations to "Russ" (e*g., "You 
should be a writer")^ but when doing so^ more users mentioned occupational 
clusters such as medicine^ education, etc. (U, 16%; NU, 7%) and fewer 
job titles; thus, the users tended to adopt GIS' s occupational cluster 
concept. In general, users were more concerned about training and school" 
ing and made a greater proportion of statements about "Russ'"' high school 
background. Non^users more frequently made statements about ("Russ'") 
hobbies and salary expectations. 



To sunsQarige the results of the two occupation questionnaires, it is 
evident that, in tems of descriptive skills and organisation of infoma- 
tion, CAG users produced ^re, and more varied, occupatlonally related 
information than did non-users. On the "self" version, users generated 
more GIS-related and job-related categories than non-users. They em- 
phasized the schooling or educational requirement of occupations to a 
greater extent than did non-users. On the "ideal" instrument, users em- 
ployed morm non-GIS-related (and GlS-related) and personal-oriented 
information than the non-user group; much of the difference is accounted 
for by users' interest in schooling, other training, and background 
academic characteristics. Looking at teth Instruments, there is a sugges- 
tion that users are somewhat more responsive to the specific wording of 
the directions than are non-users* In addition, without Judging which 
categories are "better," it Is clear that the advantages to users reflect 
the constructs and content of the Guidance Information System* 



Decision-Making 

We hypothesised that students who had had CAG eKperience, as compared 
with non-users, would more consistently recognize and apply the principle 
that as the number of limits placed on a choice increases, the number of 
options decreases. Before considering the outcomes of the Decie ' on-Making 
test, let us eKamine the ways in which users are exposed to the S 
decisionmaking or "narrowing" principle. The Student Study Gxiidu directs 
the uaer to input requests for information in sequence from most to least 
important (although it does not explain why this should be done). In the 
classroom orientations we observed, there was little or no emphasis placed 
on the narrowing logic of the TSC system. The primary and in most instances 
the only way in which the user can '-learn" or infer this principle is by 
examining the input-output interaction^-either at the terminal as it is 
occurring or by studying the printout; in either instance this may be done 
with or without additional help from a staff member. This narrowing logic 
obtains only when GIS is used for searching ; in descriptions of a specific 
college or career^ no such logic is apparent. 

The narrowing or inverted pyrmid principle seons crucial to other, 
higher-order decisionmaking processes. To estimate its impact on students, 
we developed a siK-lt^ multiple-choice Decision-Making test. Five items 
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involved the narrowing principle; the last it^ asked the respondants to 
EBlect the abstraction of the principle operative in the other items. 

Table 15 shows the results « As with the Vocabulary test^ there was 
no statistically significant differential advantage to users in any par-^ 
ticular school (use X schools interaction^ not significant). Therefore^ 
variations raong schools in students and In CAG methods or procedures did 
not give users in one school a greater advantage than users in any other 
school. 



TABLE 15 

l^AN DECISION -M^ING SCORES FOR USERS AND NON^USERS^ BY SCHOOL^^ 



School 


U+NU 




Users 


Non 


-Users 


Mean Advantage 


N 


Mean 




N Mean 


N 


Mean 


to Users 


A 


84 


3.0 




46 


3.6 


38 


2.2 


+1.4 


B 


87 


3.4 




41 


3.8 


46 


3.1 


+0.7 


C 


92 


3.9 




33 


4,4 


59 


3.6 


+0.8 


D 


69 


3.9 




38 


4.3 


31 


3.5 


+0.8 


E 


87 


4.1 




50 


4.2 


37 


3.9 


+0.3 


Schools 


419 


3.7 




208 


4.0 


211 


' 3.3 


+0.7 



"^Maximum possible score is 6. 



However^ users performed significantly better than non-users-^aa also 
was the case with the vocabulary test. The overall mean score for uaers 
was 4.0;, and for non-users 3.3; the mean advantage to CAG users was sig- 
nificant at the eOOl level. The same direction of difference prevailed 
at each school-^-users performing better than non-users. It is interesting 
to note^ however^ the variations in magnitude of the benefits of CAG use^ 
in relation to what we know about the students^ the schools^ and the 
approach to implementation. 

Although there is no statistical advantage to users in any particular 
school^ the schools* differences between users and non-users may still be 
of interest. School A^ which was poorest in average Decision-Making 

^An example of one sanple itan and the last item follows- Arnold wants 
to buy a blue compact car that costs less than $4^000. Larry wants to buy 
a blue compact which costs between $3j500 and $4^000, Who will have more 
cars to choose from?" (Arnold; Larry; both will have the same number of 
choices.) 

"What rule or principle do you think was involved in the five ques- 
tions? (1) The more restrictions that are involved In making a decision^ 
the fewer possible choices there will be; (2) At least two restrictions 
mist be considered in making any decision; (3) The more restrictions that 
are involved in making a decision^ the more possible choices there will be; 
(4) The fewer restrictions that are involved in making a decision^ the, 
easier it will be to make a choice," 
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scores, and academically poorest of the five schools, had the largest 
U/OTJ difference (+1.4)"a difference nearly twice as great ai in any other 
school- It could be argued that since School A students were lowest in 
initial levels, they therefore had a greater poaslbility of demonstrating, 
on this ilx-lt^ test, the benefits of CAG use; whereas itudenta in the 
other ichooli were already closer to the test ceiling, and thus had little 
^leeway for dMonitratlng such score iinproveEients, 

Another possible eKplanation for school differences in mean advantage 
to users might lie in differences in CAG procedures employed in the schools 
We know that students in School A received much more individual attention 
and did many more iearches than, for eKample, students In School E, where 
the overall (U4NU) mean was highest but the U/NU difference smallest « 
Although users in School E benefited from CAG, they benefited least. 

These comments suggest that the Decislon'^Making reiults reflect in 
part varlatloni in sohDol approach in implCTaentation, particularly in the 
CTployment of the search aspect of the systCTi. A further Inference to be 
drawn from these findings is that students poorer in decisionmaking 
ability might benefit most from CAG search use* A definitive test of this 
hypothesis would require an Instriment with a higher celling (more itans), 
and independent control measure of Individual ability, and assessment of 
decision-making before and after CAG searches* The prediction would be 
that students of lower ability would make larger gains in their decision- 
making scores as a result of CAG search than would students of higher ini- 
tial ability* Because of School E'a results^ a test by grade level is needed 

Whether other variables concerning CAG use might be critical is 
uncertain; time did not permit more detailed analyses of the Information 
that la available* It would be very Interesting to reanalyze the data we 
collected, In relation to the Declsion4laking and other Impact tests, 
looking at the many possible cross ^comparisons* For example, did the 
topact test scores of users depend on which file was used, whether the 
eKposure was direct or batched, and whether the student engaged in a 
search or description? Or were scores related to the time lag between 
the use and the test? Answers to the numerous questions of this kind 
might be very enlightening. Even without these answers, however, one 
conclusion is clear: Users of GIS, not separating out circumstances of use^ 
Including the kind and anount of staff assistance and attentlon--have 
batter knowledge of the decisionmaking process than do non-users* 



In considering the Implications of the Impact test findings. It Is 
important to bear In mind our asstmiption that ability was randomly dis- 
tributed across the user and non-user groups, and thus does not in itself 
differentiate between thra* We have little data about other differences 
that might have existed between the user and non-user groups. In particu- 
lar, we have only a smattering of evidence about the question of self- 
selection of the users-^and this suggests that self-selection as reflected 
in student recruitment differs OTKsng schools. Self-selection, however, 
does not account for all user/non-user differences. There are other 
important factors that have Influenced the outcomes* Not uneKpectedly, 
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the reiults on Vocabulary and Decision-Making reflect both the general 
acad^paic levels of the individual schools and the unique procedures they 
developed for using the computerized systm* 

The findings of the impact measures have striking internal consis- 
tency. It is very impressive that on each one of the six cognitive 
measures^ users performed better than non-users. Considered slK-at-a-time^ 
the benefit to CAG users is very drmaatic. In viewing these findings in 
detail^ consider thati 

Approximately half the user group tested had only in- 
direct experience with the computerized syat^^ thrDugh 
reading the materials (including the prifit^at)^ and/or 
possibly discussions with a staff mmnber* 

Approximately one-'third of the uses were to obtain 
descriptions of specific colleges or occupations. 

Approximately half the users had had their only CAG 
eKperience from 6 months to a full year before these 
Impact tests were adminiateredp 

Approximately a third of the users had had experience 
only with the occupational file^ conceded to be the 
weakest of the three major GIS files • 

Differences between users and non-users in preferencas 
for various college and career informational souraes 
were small and^ more importantly^ little related to 
their performance on the cognitive impact measures. 

Epitomising the details in major generalizations^ it is apparent that: 

•On every cognitive measure^ GAG users outperformed stu- 
dents who had not had CAG experience. Users have a 
better understanding of the terms and concepts coomaonly 
employed to characterise colleges and occupations^ and 
they are better able to deicribe and organize information 
in these areas. They are also superior in applying the 
implicit GIS narrowing logic to other decision-making 
problOT-solving situations « 

It is clear that much of the advantage to users can be attributed to 
experience with the Guidance Information System^ since much of the user 
advantage is in terras of information In that system. Notwithstanding the 
finding that GIS contains much information about colleges and occupations 
of Ijflportance to students^ from about a third to a half of what they want 
to know is not there* An improved system should build on and incorporate 
this desired infomation^ which is available. Further^ the fact that such 
a systOT has demonstrable impact on students in terms of affecting what 
they know about colleges and careers makes it essential that educators 
becOTie involved in devising a perfected syst^. 
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CHAPTER IX 

PARTICIPANTS' REACTIONS AND DISSEMINATION ACTIVITIES 



The Intent of this chapter la to put back Into this report on 
the GAG project a sampling of the excltment and enthusiasm It generated ^ 
by describing the responses of the participants and the Interest the pro- 
ject aroused both Inside and outside the New York City public school 
iyatem* 



Participants ' Reactions 

Aa we discussed earlier^ from its very Inception the principals of 
the five pilot high schools afforded the project a hospitable reception. 
Perhaps of more significance than their documented reports were their 
actual deeds of support. Each assigned a llaison(s)^ generally acknow^ 
ledged to be among the better faculty mrabers. All found or made space 
aval lab le-*not a small feat in schools with utilization rates of 165%. 
They made extra supplies obtainable^ Including postage for students* 
letters requesting additional information. Moreover^ they established 
the kind of climate that enabled the liaisons to use classroom time for 
orientation sessions^ and facilitated the flow of students to and from 
the tertfllnal room. The administration provided the utaiost cooperation 
for IRDOE's testing progrMi as well^ which often took entire class 
periods at several different times in the 15 months. 

At the end of Period 1^ the administrators' enthusiasm was at its 
highest. The principals wanted the progran to continue and to be expanded 
to all grade leveTs with more on^'line time. One way of estimating GAG's 
value was to ask principals to estimate its worth in monetary terms. 
IRDOE had calculated that^ under the Period 1 arrangCTients and rate of 
utilisation of the systOTj the cost per student user approximated $1*66-*-^ 
eKcludlng all personnel costs; three principals valued It at $4.00^ $5*00^ 
and $9.00 per student* The principal of School D said^ when pressed^ 
"For my son It would be worth $50.00* But hypotheticallyj if we [l.e.^ 
the school] had to pay from our allotaent^ 1 would assign one unit [the 
equivalent of one full-time teacher] to CAG**.*" 

One year later (in February 1975-*-Perlod 3) extr^ely positive 
feelings were still evidenced in all schools except School where in- 
terest and usage had dropped. The School C principal felt that his 
school's limited use was a function of the fact that "the novelty for the 
staff wore off"; because their students had "access to a variety of 
college informatidnal sources other than CAGj" and the "career file is 
confused^" there was a decline In motivation and use* 

The other principals remained excited^ beginning to coment on GAG's 
serendipitous effects. Thus^ for exmiple^ the principal of School D said 
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he "wlihed there was a procedure to keep the syst®a in our schools" since 
"parental comunity^ the students^ the staff^ and the a^lnlstration agree 
that this tool provides us with information that would be very difficult 
to obtain in any other manner." School E's principal^ likewise^ described 
Its broader effectsi "CAG has had almost a galvanizing effect.,, for the 
first time in a decade there have been no comunity problras* , . the PTA has 
written a note of appreciation* . .both parents and the comunity seem 
satisfied with the College and Career Office.'* The principal at School A 
notad that CAG "caused a change in a school like — ; for the first time 
in [their] experience, students [Initiated contact by] trickling in th^- 
selves asking to use it." 

The liaisons, again with the exception of School C, retained their 
vary high level of zealousness throughout the project -s duration* They 
too, towaVer, bacme more cognisant of CAG's indiract effects, as a 
s^la of responses in Period 1 and 3 will Illustrate* 1 In Period 1 
School C's llaison(s) felt that the "educational raqulrementa in the occu- 
pational file presented a form of reality testing for students" and cited 
axample^ of students who revised their goals to better accord with their 
acadOTic ability* The liaison<s) indicated that "the computer effects 
have bean positive. ** where students had thought only of working in the 
school systan with retarded children, they found from the file that there 
were posslbllitias in hospitals. Institutions, that they had not even 
thought about before*" By Period 3, School C was "enjoying our experience 
with the computer* The students are delighted with the instant responees 
and tha long list of colleges and careers the computer gives thCT**.. 
Tha job of the counselor is made easier, since GAG use means students miat 
do some [advanca] thirfcing about goals/' Althciugh the experlanca "has 
been pleasant," the School C liaison continues, "the system is too costly-- 
useful as it may have been as a motivating factor and as an informational 
resource*" 

Early, tha School E liaison noted that with GAG "the students feel 
that the school is taking a greater Interest In thm*" Later In Period 3 
she said: "CAG continuas to sttaulate interest in four-year and two-year 
colleges and hundreds of occupational fields. It has been accepted by 
guidance counselors and other savants of tha school conmunity* It has 
axpanded students' horizons* It is a powerful weapon in toprovlng conmunlty 
ralattons* In the light of this outstanding success achieved by GAG*** [we] 
feel thara will be a sarlous loss of services to tha student body if there 
is any interruption in the availability of this most needed pildance tool," 

In Period 1, the School D llalsonCs) noted that "GAG has provided us 
with the opportunity to meet many mvm Juniors at an earlier time and they 
have begun to become involved with their own futures earlier, parents are 
aware of it and have expressad interest in [CAG*s] role in planning for 
the future* Students scan their own printouts with interest--and see wider 
horizons via their friends' printouts*" Despite the decrease in usage by 
Period 3^ the liaison indicated that the "staff have unlfomly benefited 
from having 'Alphonsa' — he provides a marvelous resource of Information for 

S'rom tha Liaison Questionnaira, administered in May 1974 (Period 1) and 
April 1975 (Period 3). 
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^ counselors. The students profit from the interaction—the computer- 
oriented approach iparks their interest and propels then to become mora 
aware of what they might begin to do* It also enablea the counselor to 
spend more time with st^ents in guiding and advising them*" 

School with an eiirollraent of more than 6,000 students^ was the school 
/ TOSt attuned to the indirect benefits of CAG. The lialsonCs) in Period 1 
reported student users "who caoae to ask [the lialson(s)] about college^ and 
about their general problaiia at school and at home." In her TOrds "There 
Is an awakening of Interest in their ftiture^ some growth in decision- 
making abilities p CAG brought ui into contact with confused students who 
cannot Mke choices*. *so we [CAGl becme something of a referral agency*" 
For this liaison^ "GAG is most useful as a reality^testing and decision^ 
making aid**,, »^ The most "satiafying aspect was when students* choices 
ware brought ^re Into line with reality by their own evaluation of the 
Information on the printout." * 

In Pejrlod 3^ the liaison (School A) noted that "use of CAG has beconie 
more creative with thR passage of time anfl [thus] more adapted to the needs 
of the individual student." GAG becme more Integrated with the school^ 
wide operation^ and "counselor referrals are steady^ and individual teachers 
approach [the liaison] and ask for appolnttients for their classes. Students 
are not threatened by CAG and after using it are more willing to see college 
advisors^ guidance counselors^ and other 'school helpers*'" 

To some extent^ according to these liaisons^ the indirect benefits of 
CAG frequently outweighed the direct effects. The opportunity to interact 
personally with adult staff proved invaluable to many students (e.g.^ the 
student who was "saved" from becoming a "dropout" of a four-year college 
to which she should not have applied in the first place). Another very 
drfflnatic ex^ple involved a senior who was having trouble specifying 
characteristics for a college search; with mintoal prodding^ the student 
stated that her real probleai was separating from her ''best friend" after 
high school* Other exmples abound^ especially in School A^ where the 
staffing; the large B±zm of the student body^ and the overlapping sessions 
were such that few students had an opportunity to spend any time with an 
adult professional in a non-classroom setting. The School A liaiaon(s) 
reported that CAG students would very often stop them in the halls to ask 
all kinds of questions- according to the lialson(s)^ CAG Increased the 
desire for ^Idance' In the school* 

Liaisons reported a change In how students perceived the college 
office; according to one^ the office bec^e a place where "kids who weren't 
going to college could come." And they said more students were going to 
the college office^ with the advantage that advisors could see students 
earlier in their high school careers. In general^ the liaisons agreed that 
GAG sttaulated questioning. Students asked more questions becauie"wlth 
CAG it is easy to do soy they get Immediate responseSjp and they ask these 
questions at an earlier stage In their lives when It is easier to make 
changes . 

Liaisons noted other indirect effects. Slow readers tried to keep up 
with the teletypewriter and its unfamiliar format as It was printing answers 
to their requests. Students made use of college and career materials In the 
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school library. Thay «rote postcards requasting occupational information 
froffl the iourcea listed in the printout* Liaisons felt the studenti were 
bacomlng increasingly aware of the relation be^een their high school 
CDuraea and their future plans , 

The teachers and ^idanca counsalors who responded to the School 
Personnel Questionnaire given near the end of period 3 unaniniously agreed 
that GAG was a useful tool for advising students about collegei or careers. 
The reasons they gave varied. One counselor said that it reduced the 
tachnical aspects of his Job^ while another, respondent viawed GAG primarily 
as a nsDtivatlonal tool* Its use was found to "give precislori to the gen- . 
era! ideas of many students and to remove incorrect riotlons*-' A grade 
advisor Involved In GAG noted that it "compels students to make decisions"; 
one collage advisor wrote that GAG "helped studeiuts make more realistic 
choices*" 

Atoost all respondents noticed increases In the volme or changes 
in the type of student interest In collage or career information. The 
changes described included increased student interest in, and consideration 
of, a wider range of occupational and college infoCTiation* While most 
respondents realised they could not prove that GAG caused the observed 
changes^ it Is still Interesting that at every school^ faculty involved In 
the project noticed changes in studenti # 

The questionnaire to librarians^ also administered near the end o£ 
Period 3^ examined how aware they ware of GAG their reactions^ and whether 
they perceived any changes in student requests for infomatlon* All thei; 
librarians had heard of GAG from either teachers or students. Only the ft^o 
School 1 respondents had actually seen a deattonstration^ - but most others 
had seen many computer printouts. 

Overall^ the librarians had a positive Impression of GAG; the one 
neutral raspondent^ from School A, said she was not sufficiently ac- 
quainted with it CO judge i The librarians (eKcept at School A) noticed a 
change in the volume or type of interest students expressed in career and 
college information. One indicated that students came to the library with 
more specific focus ^ while another simply reported that students asked a 
greater variety of questions about jobs and colleges* Three librarians 
felt that the changes they noted were related to GAG- 

In sunmiary^ it is apparent that the other members of the staff-- 
librarians^ ^idance counselors^ and involved advisors and teachers at 
each schools-concurred with the liaisons In their positive impressions of 
CAG^ as indicated by their reports of increased student interest in col- 
leges and careers and In a much greater variety of colleges and careers^ in 
motivational effaets^ and In greater student facility in making decisions 
and choices. 

Dissemination Activities 

parents of students in all the pilot hiyh schools have been exposed 



School EVs terminal was located In the library. 
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to GAG* Indirectly^ llal^oni encouraged students to take their printout 
hams and discuss it with their fmily. Directly^ all schools discussed 
GAG at Parent Association meetings in one or several sessions^ and four 
achdols conducted actual dCTionstrations o£ the systan* It is estimated 
that about 400 parents sa^ CAG in operation. Besides seeing it presented 
at public Parent Asioclation meetings^ parents had individual Involvraent^ 
usually together with their child in a discussion with a counselor or 
teacher* The reported nuEiber of parents so Involved ranged from 3 to 20 
per school. The principals and liaisons reported that parents and com- 
munity members were "impressed and happy" with CAG^ "s«*,pleased that their 
children are thinking about the future" and "pleased that they are taking 
their classwork more seriously." 

Principals reported that parents of children attending other high 
schools phoned requesting that their children be pemltted to use the 
computers A large nwnber and wide range of young people from other sec- 
ondary schools used the Guidance Infomatlon Systm; they were referred 
by friends at the pilot schools^ parents^ teachers^ and counselors. And 
they Included siblings^ and relatives of faculty^^ The liaisons esttoated 
that altogether^ between 200 and 300 students from other public and private 
high schools reoelved information^ either directly in a visit on an on-line 
day^ or through batching. The pilot schools hosted sm,ill groups of these 
visiting students^ handled telephone reqiiasta^ and did some followup over 
the phone^ both for Individual students requesting infomation and for 
teachers r 

College students also used the prograon. Past graduates of the pilot 
high schools returned when they heard about it and asked to use it. The 
liaisons reported fulfilling requests for information from CUNY students 
in at least three community colleges and one four-year college. Demon- 
strations and requests for information cmm^ttom the New York State 
Employment Service^ IBH^ Hewlitt Packard Data Systras^ New York State 
Bureau of Labor Statistics^ CUNY Office of Amissions Services (which was 
so taken with the concept that It subsequently applied for and received 
federal funds to establish Computer -^Based Equal Opporttffilty Centers for 
SK-of fenders^ eK-addlcts^ and the handicapped disadvantaged population)-- 
and from Boricua Universidad. 

Demonstrations were conducted for various local groups and agencies 
representing bureaus and offices of the New York City Board of Education^ 
First National City Baidc^ and the New York State Education Department. 
Included were (from the New York City Board of Education) the Director and 
staff of the Bureau of Educational and Vocational Guidance together with 
the High School Supervisors of Guidance and the Director of College 
Guidancei representatives from the Office of the Deputy Chancellor, Office 
of Career Education, Burean of Business and Distributive Education, Bureau 
of ManagOTant information and Data processing; and the Advisory Council 
for Occupational Education. The Economic Development Council participated 



ERLC 



1 

.These Included students from Tllden High School, Wllllftn Cullen Bryant 
High School, George Washington High School^ Brooklyn Tech^ the Mary Louis 
Acadray, the Hunter College Campua School, Great Neck High School, West 
Hempstead High Sctool, and 7 or 8 public secondary schooli In Queens, 
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in deotonstrations^ as did representatives of the Explorer Scouts and staff 
from the New York State Project to Improve Gareer Education. 

At the New York State Education Department leval^ GAG was seen by 
representatives of the Bureau of I'wo-Year College Progrmis and the Divi- 
sion o£ Occupational Education Supervision (who^ in 1974 began funding the 
companion project at the conmunlty college level); the Division of Adult 
and Continuing Education; and the Research Coordinating Unit* 

Requests for information about the project were received from the 
State of Hawaii^ Department of Education; West Point; the National Insti- 
tute for Career Education and Counseling; the Minnesota Educational 
Computing Consortlim; Project IR^ (New York City Mayor's Office); the 
Newark (Delaware) School District; the Queens Men's House of Detention; 
the New Rochelle City School District; Station WOTT-^TV (Annendale^ 
Virginia); the National Council of Jewish Women; Harahe-Rotoan and Druck, 
Inc* (Chicago-baied ITT Educational Services); Syracuse University; 
Brighton High School (Rochester^ New York) ; the New York State Education 
Department Office of Vocational Rehabilitation; the Alleghany (Pittsburgh^ 
Pennsylvania) Intermediate Unit; Long Island University; and from the 
Careers Reiearch and Advisory Center (Cambridge^ England) * 

Stimulated by a Press Conference on February 25^ 1974 at Louis D* 
Brandels High School^ articles appeared in many local newspapers and 
magazines; the televiilon media carried the story on MC Eyewitness News 
(2/27/74^ 6i00 p^mOj NBC Evening News (2/25/74^ 5:00 p*mOj and on 
Channel 9 News (2/25/74^ 11*00 p#m.)e It was reported In the early and 
late edition of the New York Dally News (February 21^ 1974); In the 
New York PoJt (February 26, 1974); In the New York Teacher (March 10^ 1974) 
In Parade Magazine (March 31, 1974) ; in Business and Society (March 19, 
1974); twice In the Staten Island Advance (Febniary 20, 1974 and October 6, 
1974) 5 in a newspaper In Tucson, Arizona (1974); and In Senior Scholaitlc 
magazine (April 18, 1974). 



It is difficult to esttaate how many people have been exposed to the 
concept of" computerized guidance "through the CAG project in the New York 
City high schools* The nature and number of the inquiries we at IB^E 
and the staffs of the pilot schools have received suggest that locally 
and nationally there Is a high level of interest in the potential of the 
Guidance Information SystOT and in other similar systmii* 
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CHAPTER X 



SimARY^ INCLUSIONS' AND REC0>MEI®AT10NS 



This chapter concludes IRDOE's study of the feasibility and impact 
of the computer-assisted guidance project which operated In five New York 
City public high schools between February 1974 and June 1975. This report 
attempted to sutranarlEe the myriad of detalls--dlf ferences in implementa- 
tion procedures and atylesj and individual school variations tn results-- 
that characterised the danonstratlon project. This sunmiary chapter seeks 
to focus on those aspects whleh"-on the basis of our experience-^-seem 
crucially related to the success of similar ventures in the future* 

That the CAG project was successful is not questioned; the enthusiasm 
of the participants^ staff and students | the number and variety of students 
who were served; the Integration of CAG with the schools other educational 
functions 5^he positive effects of usage on students' knowledge^ organisa- 
tional^ and decision-making abilities; the Indirect benefits to the 
schools' counseling and advising services; and the nunaber of new projects 
generatad as a direct spin-off from this one are ample evidence that 
serious consideration needs to be accorded to CAG. 

Moreover^ the CAG project is funl Everyone we cme Into contact with 
at the pilot schools and the hundreds of outsiders who saw it In action^ 
found it a pleasurable experience^ Including ITOOE staff. The highly 
enjoyable response CAG evokes should not negate the positive effects on stu- 
dents we have documented. All too often^ new "specially funded'* projects 
report that staff and students liked It^ although there "werR no other 
noticeable effects*" The CAG project is an example of an enjoyable 
deasonstratlon that worked* 



SuMary; Background and Objectives 

Funded by First National City Bank and the New York City Board of 
Education's Division of High Schools^ In February 1974 IRDOE Initiated a 
computer-assisted ^idance project in five high schools selected by the 
Borough Superintendents --Louis D, Brandelg in Manhattan^ Herbert H* Leteian 
in the Bronx^ Midwood In Brooklyn^ Francis Lewis in Queens^ and Port 
Richmond In Staten Island. Initially^ the project was to be operative 
from February to June 1974 (Period 1) and from September to December 1974 
(Period 2); the' first eKtension--f rom January to June 1975 (Period 3)-- 
waa In response to early findings and the schools' positive reactions. 
A contractual modification for Period 3 enabled IRDOE to assess the direct 
affects of the computer experience on student users. The purpose of the 
geqond eKtenslon for the school year 1975-76 (see Postscript) was to 
permit 'the concerned agencies to discuss ways to modify and/ or maintain 
the project in anticipation that all special danonstratlons^ no matter 
what the findings^ must eventually stand on their own. 
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The initial objective of the pro ject^ targeted at students in 
grades 11 and 12^ was to answer the question^ "Is an automated, inter- 
active Information retrieval systOT a feasible way to provide students 
with college and career information needed to make plans for their 
future?'' The feasibility question consisted of issues related to instal- 
lation^ maintenance, and functioning of the hardware; adequacy of the 
software; scheduling, and the assessment of the utilization of time; 
staffing patterns; and extent of usage by students, Also of interest was 
whether CAG use affected the number of students applying to colleges, the 
types of colleges applied to, and the rates of acceptances; and, the 
effects on career choices of students who were not college bound. 

In Period 3, the evaluation objectives were expanded to ascertain 
the effects, if any^ of CAG use on students' knowledge of college and 
career information, on their ability to conceptuallEe (i,e,, organige) 
this information, and on their decision-making ability* 

The computerized system used in the demonstration was the Guidance 
Information System of Time Share Corporation, a proprietary company which 
developed^ maintains, and markets the system for national distribution at 
the secondary school level, GIS consists of four data flles-^one con- 
taining information on about 1600 four-year colleges, a file of informa- 
tion on about 1000 two-*year colleges, information about 1300 occupations 
selected from the D aO .T. ^ and a scholarship and financial aid information 
file. The scholarship file, though sorely needed, was little; used and 
for all Intents and purposes can be considered existing in a prototypical 
state. All 3 major files can be approached in two ways: to obtain a 
description of a prenamed college or career (description), or to obtain a 
list of colleges or careers that meet the user's input Specifications 
(search)* In conducting a search^ the user Is exposed to the narrowing 
logic on which CIS is based^-napnely, each additional input restriction 
narrows the remaining opt ions , All the information necessary to operate 
the system is included in the User Manual and Student Study Guide, 

In the New York City project, to access GIS, the user sat at a 
teletypewriter terminal (one per school) connected by phone line to TSC^s 
computer, (S)he types in his/her instructions, employing letter and 
nifflierical codes. The computer responds by typing back on the keyboard. 
The interaction simulates a rudimentary conversation* The tnteractlon 
can be direct with the student present, or indirect where a terminal 
operator processes the student's request in his absence. In this latter 
mode of use (batching), there Is a delay between the time the student 
submitted his written request for information and the time he received 
that information* In direct interaction there is no such delay and, 
moreover^ a student can modify his input on the basis of output he Im- 
mediately received. Carbon paper was provided so that each student could 
retain a copy of his interactioi^ with a carbon for staff use. 

Each school was assigned one day a week in which it had unrestricted 
access to the computer from 8* 00 a ,nu on(theoretically until the following 
morning at 6:00 a.m.). In Period 1, on-^line days were rotated so that, 
for eKample, in the first month School A was asslgnad Mondays, in month 2 
Tuesdays^ and so on* In Periods 2 and 3 a set day of the week was 
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assigned to each echool. starting in Period 2^ IRDOE initiated a similar 
project at the coCTnunity college lavel--oiie of the unanticipated benefits 
of the two companion projects was an incraaae in the flexibility of 
scheduled time* 

KDOE*s grant from Citibank covered all hardware and software costs^ 
and costs aiiociated with project management^ monitoring^ and evaluation* 
The Board of Education's contribution covered the salary of one full-time 
teacher equivalent at each of tl^ five schools^ and the telephone service 
charges « 

The school administration assigned one or more mmbers of the staff 
to act as project liaison(s). In 4 of the 5 sctools the teaching equiv- 
alent salary was split either mong two profesiionals^ and/or between 
professionals^ paraprofessionals^ and secretarial aides. The liaison was 
responsible for in-school publicity^ and student recruitment and orienta-. 
tion| operating or supervising the operation of the terminal| student 
followup after usej coordinating CAG with other school functions/ and 
maintaining and expediting the evaluation records and test and question- 
naire administration* The on-line day was largely devoted to helping 
students at the terminal; most of the other duties were performed on the 
days the school did not have access to the computer* 

The participating schools varied on many dimensions^ some of which 
were related to school-to-school ifferences in outcomes* They ranged in 
size from School the smallest^ to School A which enrolled nearly three 
times as many students. School E was built about 50 years ago^ while 
School B was newly opened and did not have a full graduating class until 
June 197S. About 70% of School A*s students were reading two or more 
years below grade level^ with the range at the other schools from 16% to 
2e%» In tenns of academic climate and general SES level^ Schools G and D 
served a relatively affluent comQunity and had a reputation for acad^ic 
eKcellencei Schools E and B were largely blue collar^ and School A was 
the poorest^ and with the largest proportion of non-white students (90%), 

The schools* involvement in career education emphases differed as 
did the extent of their college advising services* These tended to co- 
vary: School B was the most career-oriented and had the least developed 
college advising services; Schools C and D had well-established college 
offices but much less well-defined career aiivWory services. School E 
had a combined career-college office* School A had a modest college 
office and little involvement in career education. 

Although all schDols tended to be overcrowdad and short-staffed 
(especially in Periods 2 and 3 when there was a .^:eneral worsening of 
economic conditions citywide)^ they made space for the project and as- 
signed liaison responsibility to their "better staff members.'' A grade 
advisor was primarily responsible for GAG at School A and the terminal 
was located in a rocm within the collage office* School B hired a full- 
time piidance intern and placed the terminal in proximity to Its career 
advisory programs* At School C the terminal was In the college office 
and primarily supervised by a college advisor* a similar arrangement 
obtained at School D. School E's CAG operation was under the supervision 
of the college-career office advisor^ and the terminal was located in the 
fourth floor library. 1 1 Q 
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In March 1974j lEDOE collected baseline data on 11th and 12th graders 
plana with regard to education and carears^ the factors Involved in thetr 
declBionB^ and on the quality and quantity of their In-school interaction 
with couniellng and advising staffs 

Moat students Intended to go to eoilege^to a four-year rather than 
a two-year or vocational Instl cation,, In large tneaiure^ this reiulted 
from CUNY's Open Admissions policy. Almost every student could state 
occupational goals^ although there were marked differences In level and 
epeclflclty ranging from cytoteCiinologlst to plumber to painter* Twelfth 
graders^ as compared with eleventh gr&deri, tended to be mora restricted 
In their occupational goals— iuggegtlng a focusing or narrowing of choice 
as "cloier*-to*declsi0n-tlme" approached. Most students Indicated that it 
wasthelr own Interests and abilities that conitltute the major Influence 
on their future plans— far BMjre important than friends^ teachers^ coun- 
selors^ or written or media materials. 

Most students surveyed said they had seen a counselor or advisor 
at least once for Individual consultation and had received help^ especially 
with their future plans for continued education- Sizable proportions of 
students reported wanting more help--especially with their college plans* 
Those who Indicated wanting more help with occupational planning seemed 
Interested in the educational component of careers i this Interpretation 
is supported by the findings of the Impact study, Thus^ when the CAG 
project started^ students evidenced a need for both career and college 
planning^ whether specifically articulated or not* 

IRDOE Initiated the project by conducting school orientation and 
staff training sessions, Infomal "training" and mutual learning con- 
tinued with each of the 97 man-days of visits made in the 15 months of 
the project's duration. We designed several instruments and data collec- 
tion techniques to collect time data^ malfunctions^ numbers and grade level 
of students who participated^ flle(s) uied^ and mode of Interaction, 
Questionnaires were administered at regular Intervals to liaisons and 
principals^ and to the guidance staff^ school librarians^ and other In- 
volved teachers. Students were tested In 1975, College applications 
made by the 1974 a^ 1975 graduating classes were analyzed. Records were 
kept describing the quality of GIS and Independent assessments were made 
on the accuracy^ recency^ and usefulness of the systrai. Anecdotal reac- 
tions & evidence concerning serendipitous benefits and problm areas were 
preserved* 



Suianaryi Conelusions and RecqiTOendatlQns 

Many of the specific recoOTnendations follow directly from the stated 
conclusions. Instead of restating these in the •should, , ," form^ in 
this concluding section we will concentrate on two reconmendatiDns i Implemen 
tatlon and modification of the software progran* Throughout this report 
we have also stated or In^lled that there are still some very crucial 
research questions that should be addressed"-elther prior to or concurrent 
with any continuation^ expansion^ or change with regard to computer- 
assisted guidance in the New York City public high schools. 
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A total of 4137 different students (users) in the five high schoQis 
used CAG a total of 6789 times (uses) in the 15 project months* For an 
Individual . student^ use can consist of one or more of the following ex^ 
perienoesr 

Review of the Student Study Guide (or a modified facsimile) 
in whole-class orientation sessions^ and/or in small-group 
or individual sessions; 

Single or repeated use of one or moVB of the CIS files; 

The conduct of a search^ or a description^ or both; 

The presence or absence of the student at the terminal when 
his request for information was being processed; 

Receipt and/or review of a copy of his printout; and 

Followup activities* 

Each user averaged 1*6 uses; if these had Included his presence at 
the terminal^ the average length of time he spent at the terminal was 11 
to 26 minutes — a very brief experience in ©ne*s high school career. About 
62% of the students used GAG once; 38% returned two or more tteies* There 
were more than 130 Individuals who returned 5 to 13 times* 

Fof'all schools*' and at each individual school^ the total number of 
students served was greatest In period 1 (total users^ 2274) and least in 
Period. 2^ the Fall SCTester (total users* 1413), For each period^ the 
separate schools maintained their respective positions^ with School 1 con- 
sistently serving the MSt students (total alt periods^ 1167) and School G 
the fewest (total all periods, 498), 

Taking Into account the difference in period durationj the number 
of scheduled days, and the availability of extra time^ use In Periods 2 and 
3 was approxtoiately half of what could have been eKpected on the basis of 
the least conservative prediction* The decrease was a result of several 
factors^ including the fact that the novelty wore off^ the topetus to 
serve as many students as possible declined^ and schools began to use the 
system differently and with better Integration with schoolwlde affairs. 
The priMry factor^ however^ appeared to be economic--despite the con- 
tinued allocation of ataff at the Bamm level^ actual staff time was reduced 
considerably In Periods 2 and 3^ especially in Schools G and D* 

In Period 1^ the combined schools' average number of uses per day was 
40*4^ as contrasted with 18,8 in Period 2 and 18*6 In Period 3* This 
accords with the overall decrease^ but also reflects the difference In 
availability of extra daysi In Periods 2 and 3^ the schools used more 
daySj but had fewer uses per day* 

On each on-'llne day^ however^ schools maintained a relatively con^ 
atant pattern of use; that Is^ in Period 1^ there was an average of 7,0 
uses per hour* In Periods 2 and 3 the mean number of uses per hour was 
6#3 and 7.4^ respectively, Thus^ the schools tended to process approximately 
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the sme number of requests per hour of terminal time* Differences in 
mean nimber of uses per hour is directly related to the proportion oi uses 
that involved direct Interaction. Schools E and B which tended to average 
the largest mean nMber of uses per hour (all pariods)^ tended to have the 
largeBt proportion of batched requests (all periods)* When the overall 
proportions of direct use went from 29% to 48% to 28% (Periods 1^ 2^ and 
3^ respectively)^ minutes per use went from 9 to 10 to 8 minutes. To 
conclude t 

• In the 15 months during which CAG was operational In the five 
public high schools^ a total of 4137 different students used 

CAG a total of 6789 tijnes--or an average of 1,6 uses per student. 

• The total number of users and uses wag highest in Period 1^ 
followed by Periods 3 and 2* Thus^ there Is some suggestion 
that — all other things being equal — use can be expected to be 
higher in the Spring than in the Fall semester of the school year. 

• Most students used CAG once; 38% of the total users used CAG 
more than once. Of the multiple users ^ a small number returned 
numerous tlmes- 

• The individual schools differed with respect to users and uses* 
Their relative positions were consistent over the three time 
periods; School E served the largest numbers of students and 
Schools C and D the feweat, 

9 Use in Periods 2 and 3 was about half of that in period 1, The 
primary factor was the decrease in the actual toiount of staff 
time that was devoted to the project| this reduction was related 
to worsening economic condltidns that affected staff patLT^rns 
most severely in the college offices at Schools G and 

• The availability of extra time in Periods 2 and 3 resulted in 
the schools' use of a greater ntmber of different days with 
less use on each of these on-line days* This is also related to 
staff cuts. Staff is available the same number of hours each day; 
in Periods 2 and 3, they had lass time each day, ' 

• On each on-line day schools processed apprOKlmately the sane 
niOTber of requests per hour^ 7*0 In Period 1^ 6*3 in Period 2^ 
and 7.4 In period 3. Individual differences among schools are 
related to the proportion of uses that were dlrect--the larger 
the' proportion of direct use, the fewer requests processed per 
hour of time* 

• In general^ less time may be devoted to CIS in schools which 
have a large and w^ll-established college advisory office an 
academic reputation, and from which large numbers of graduates go 
on to college* CAG may be most useful in very large schools 
and/or those with less of an histprical college bound orien- 
tation. 

• Although there is li.ttle direct evidence^ as a tentative 
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hypothaals^ had GIS (including the Study Guide) been niore appropri- 
ately designed for the City high school population as rep- 
reiented by these schools^ the non- or leaser availability 
of staff nay not have resulted in as great a decrease in 
student use* 

* * * 

The five pilot schools shared one port in the computer. In period 1^ 
lEDOE assigned each school one day a week on a rotating basis, in Periods 
2 and 3 the liaisons indicated a preference for a regularly assigned. week- 
day^ which was instituted* Apparently^ all liaisons (with the eKception 
of School B) had other teaching and non-teaching responsibilities so that 
it was easier for thm to plan their time on the sMe-day^a-week basis. 
In Periods 2 and 3 IRDOE also ran a coiiipanion project which enabled all 
high schools and community colleges access on extra days* 



In Periods 2 and 3^ schools traded fewer days anong themselves and 
used a greater proportion of days assigned (or requested) than they had In 
Period 1, In Period 1, for all schools combined^ 89% of the days scheduled 
were used; in Periods 2 and 3 the schools used 101% and 104%^ respectively^ 
of the original number of scheduled days (because they used extra days). 
While there was school^to-achool variation in the anount of time used each 
day^ all schools combined averaged 5% hours per day In Period 1^ 3 hours 
per day in Period 2^ and 2% hours per day In Period 3# With the exception 
of School E (which maintained a steady daily rate of use in all three 
periods)^ mean hours per day for each of the other four schools decreased 
from period 1 to Periods 2 and 3. 

To some extent the reduc*-ion in average hours per day^ as Indicated 
'above^ is eKplalned by the fact that the schools t©ok advantage of extra 
days with the result that scheduling half rather than whole days might be 
better--^ight maKimlze use--in those situations where the staff does not 
have an entire day to devote to the operation of the terminal. 

Although the school principals asked for more t:han one day a week 
access^ the liaisons did not; they tended rather t ad to be able to 
divide one day's worth of access time among the schuol weekdays, 

• One day a week worth of on*llne terminal ^ime appears to be an 
adequate anount for schools differing significantly in size* 
Obviously^ smaller schools can (and do) process requests for a 
larger number of students^ can can reach proportionally more 
9th and 10th graders with the ssmm amount o' time than can 
larger schools* 

• There is an inverse relationship between the ntunber of differ- 
ent days an individual school made use of the computer and the 
niffl^er of hours it used that 4^y' The greatar the nimber of 
days^ the fewer hours per day* 

• FleKibillty in scheduling the one -day- a -week equivalent is 
important. In Periods 2 and 3^ such flexibility resulted from 
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additional computer ports | without thls^ assigning each school 
two half-days may be batter than the asslgiment of one whole 
day of temlnal time. If one day a week ii assigned^ school 
staff prefer the same rather than a rotating weekday, 

m Schools vary with respect to the number of hours per day they 
use the terminals Each school^ however^ tends to be fairly 
consistent, 

s No school averaged more than ^ hours per day (the overall 
schools^ overall periods average was 3%)^ despite the fact that 
the computer was available for 22 ho^^ a day. Coupling a high 
school GAG prograQa with an evening program of some sort would 
use computer time to best advantage, 

# More time may be needed by the high schools in the spring 

semesters. Since indirect use also predominated In the ^rlng^ 
perhaps had imre time been available , a greater proportion of 
students could be present at the terminal when their requests 
were being processed . 

^ * * 

All computer activities^ including the CAG project^ suffer from 
hardware malfunctions, it is evident that most major diaruptlons In 
equipment affecting CAG use occurred in the start-up months^ and were 
related to the Installation (or reinstallation In Septmber 1974) of 
the terminals and the telephones * For Periods I and 2 the schools re- 
ported an estimated total of 63% hours of time lost due to hardware 
problans; about three-quarters of the tims. lost were In the first two 
months of each period. Most of the difficulty was with "coraminlcationa'^ 
including problems with the phone lines to the central computer (resolv- 
able by the telephone company)^ as well as problCTis related to the 
computer Itself (e.g,^ busy signals^ no signal)* In general^ the liai- 
sons rated TSG's service as "good" to "eKcellent"--wlshiog only that there 
be some way to forewarn users about central computer problOTS that could 
be anticipated. 

The perception of the severity of hardware probl^s decreased with 
eKperlencejj as the liaisons learned the techniques for handling (at least) 
minor problCTis thmiselves. Similarly^ the initial negative reaction to 
the noise the terminal makes abated^ even though nothing was done to try 
to resolve this problaas 

Within a short space of time the hardware becme peraonalized--and 
the terminal^ the computer^ and TSC-s manager of customer relations all 
received flattering nicknmes. In the 15 months during which the hard- 
ware was in the schools^ there were no security probl^s reported. More- 
over^ there was no instance of theft or abuse of the equipment^ and not 
one of the malfunctions was a result of mlstraament* 

m Thus^ frOT a technical viewpoint^ automated equipment for 
student use is secure in the high schools If normal pre- 
cautions are taken. Staff and studanta alike learn to deal 
with the hardware and malfunctions with a minimum of fuss. 
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• TSC provided excellent technological support and maintenance 
services; the project goals were furthered by the very high 
quality of systCTi and conrounlcations network functioning. 

0 A teletypewriter terminal^ although the model used was nolsy^ 
was Indeatructible and provided a itudent with a hard copy of 
both his input and the output* Its relatively slow typing 
speed enabled a student to try to read the output as it was 
being generated * 

it 'k if 

Questions related to the software components of GIS--the Student 
Study Guide^ the content of the files^ and the basic structure and or- 
ganization of the system---are somewhat more coinplex. There was a great 
deal of concern about the suitability of the Study Guide for the high 
school population represented by these five schools* All schools at- 
tffliptad one or another procedure to cope with the difficulty level^ format 
problems^ and instances of Insufficient explanations e Many of the prob^ 
1ms were handled by tha liaisons In Individual orientation with students 
In which they rephrased or redefined the written material. The schools 
also rewjrote parts or excerpted sections of the materials themselves. 
Several of the problems (e.g*^ format^ index of occupations by cluster) that 
we brought to TSC's attention were modified in their annual revisions of 
the Study Guide* 

m TSC la responsive to consianer input^ at least to the extent 
that the requested change does not violate GIS-s national 
marketability, 

• The Student Study Guide is too difficult for the New York 
City high school student^ as represented by the pilot school 
enrollees- A tentative hypothesis^ for which we ha-g^ little 
direct supporting evidence^ is that the reading level dif- 
ficulty of the Study Guide reduces the probability that large 
numbers of .students would be able to delineate and process 
their own requests for information by reading the Study Guide 
and operating the terminal themselves, 

* * 

In general^ the liaisons and our independent research supports the 
finding that the basic logic Is sound^ flexible ^ and relevant to the uses 
to which a school could make of a computeri2ed system. Three schools 
tended to use the system as a "one-^shot" experience^ while the other 
two tended to encourage repeated exploration. For all schools combined^ 
62% of the students had had only one GAG experience at the terminal^ al- 
though there was the occasional student who returned seven or more times. 

All schools liked the search and description capabllitieSj and dif- 
fered from one another in the degree to which they were s'ressed. In 
Periods 2 and 3 (for which these data are complete about two-thirds (67%) 
of the uses were searches. Almost every use by Schools B and Cj and A 



125 



f 

114 



(in Period 1) and D (in Period 2) were searches; only In School E did the 
proportion of descriptions outnimber searches by 2 to 1. The impact 
findings an the Decision-Making test clearly suggest that the advantages 
to user groups as compared with non-users is related to the proportion of 
users who requested a search^-the capability that exposes users to the 
systCTi's narrowing or "inverted pyrmid'' logic. 

Another aspect of CIS flexibility is that it permits adaptation to 
direct use or indirect use. Indirect use, because it Is faster (average 
3-6 minutes as compared to direct use which averaged 7-16 minutest and 
la not dependent, on. students keeping appointo ^=^.®..?R^?P-^fl . 

serve a greater mmber of students. Direct interaction was reported to 
be the preferred tmdm^ since it increased the opportunity for student- 
staff interaction, was fun for the student, and enabled him to revise his 
input immediately on the basis of the output* Despite the fact it was 
preferred^ its supposed advantages did not outweigh time and dCTaand con- 
siderationss In Period 1, 71% of the uses were indirect; in Periods 2 
and 3 the percentages were 52% and 72%, respectively* Although there were 
school-tO'-school (as well as period- to -period) variations in the proportions 
of direct and indirect CAG experience, these were not reflected in the 
Impact study results (neither Vocabulary nor Decision-Making), 

The final aspect of CAG fleKibllity Is that CIS Is elastic enough to 
provide an eKperlence to studants In grades 9-12 with varying Interests, 
attitudes^ and abilities. Most of the users were in grades 11 and 12 as 
a result of priorities IRDOE established* In Schools B and E, however, 
from 21% to 48% of the uses were by 9th and 10th graders. (In the in- 
stance of School B because of two factorsi no or a small twelfth grade 
in Period 1 and its career orientation which permeated grade levels* 
School Ej because of its size, was able to serve large numbers of juniors 
and seniors with sufficient time left over to serve sophomores and fresh- 
men.) The liaisons differed with respect to their opinions about who 
benefited raost and least^ — although all agreed that CAG stimilated students' 
thiiAing about future decisions earlier in their high school careers. The 
impact test findings are inconclusive with respect to grade level since 
only School E tested a substantial nmber of 11th grade students on 
Vocabulary and Decision-Making. 

m A system such as GIS offers the New York City secondary schools 
enough depth for repeated exploration and enough scope to 
process a ''one-time" request for information* 

m GIS can be used with success in grades 9*12^ according to liai- 
sons- reports; little other data with respect to the differen- 
tiated benefits by grade are available, Smilarly, GIS can be 
used with students differing in ability level, area of interest, 
and motivational level. The Vocabulary and Decision-Making 
results (on a school'^to-school basis) suggest that schools with 
greater proportions of lower ability students make the most 
drffliatic gains as a result of CAG^ although this might be an 
artifact of the low ceilings on these tests, 

• The computerized system's search and description capabilities 
permit it to be of use to a variety of students, including 
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those who just want a description of a spacific college or 
career or for those who have , more or less, defined those 
attributes they want in a college or career. Even for stu- 
dents who have made a flm decision or commitment^ CAG was 
used as reality testing of that choice* 

% Schools' own needs^ philosophy^ and size are related to their 
usage patterns* Some schools will encourage multiple uses by 
a single student; others will try to reach as many different 
students--at least once--aa possible. From the data we ana- 
Ij^Bdy i^B cannot deiremiTie to n^^iat mxteTrt 
related to the Impact of the eKperience on the user, 

e Although not tested on an Individual-by-indlvldual basis^ the 
schopl-by^school evidence tends to support the conclusion 
that eKperience with the system's narrowing logic (i,e*^ the 
system- s search capability) improves students' decision- 
making ability. 

9 Despite the professed preference for students' direct inter- 
action^ the schools tended to batch an average of six of every 
10 requests for Infomation, The proportion of requests 
that are processed with the student present at the terminal 
appears to be positively related to the ability level of the 
student body^ the emphasis the liaisons established (or 
pressures they responded to)^ and whether students have free 
periods during the school day or free time after school. 
Student apathy or forgetfulness In keeping appointments 
and pressures to provide the service to as many students as 
possible Increase the proportion of revests that are batched. 
We did not differentiate the benefits of direct vs* indirect 
use^ on a schoolwide basls^ on the Vocabulary or Decision- 
Making Impact test results* 



m Direct use^ however^ facilitates the positive involvanent of 
students with staff^ so that stronger personal relationships 
are established^ and students feel more comfortable in sharing 
school'-related and non^^school-related concerns with the result 
that better and more realistic future plans are madea All 
direct users received a copy of their printouts^ reportedly 
shared with friends and fmlly^ thereby causing a "ripple" 
effect ^^ost noticeable in the Increase in number of students 
who "heard about CAG from a friend" and dropped In to "use the 
compu ter .'■ 

m Indirect use increases the need to establish routines to in- 
sure that the printout is returned to the student and to 
arrange followup discussion about the output. 
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The liaisons rated the data files in CIS for completeness^ accuracy 
recency^ clarity^ and relevance or usefulness. Their ratings were sup- 
plemented by independent measures of these variables . The purpose was 
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two*foldi flrst^ to attanpt to separate the concept o£ computerized 
aervlcea from the specifics of the Guidance Infomatton Syst^; second ^ 
to provide a basis for the design of an optimal systan that would be 
moat Maningful for New York City students* 

In looking at all four files^ the two college files were judged to 
be fairly complete in that moat (but not all) of the four*-year and two- 
year institutions were included* Aa far as educational information is 
concerned^ the lialsons-«ref lecting their experiences witti students-- 
indicated a need for additional files. The liaisons felt it would be 
iMLSt deslrabl^^ selected graduate schools (e,g*, university 

schools of journaiism^ optometry^ etc) — especially ones offering either 
fairly unique opportunities and/or ones accepting candidates who had not 
completed the traditional full four years of undergraduate school* Of 
more intereat to thanj however^ was the need for a file of post-secondary 
vocational training schools^ Including specialised schools^ and technical^ 
trade^ and buainess schools**«particularly schools at which a high school 
graduate could obtain the training necessary for some occupations in the 
occupational file^ not always in a two-year or four-year college 

setting. Thus^ not only was there a stated need for Inatltutions of- 
fering technical'-plus training^ but a strong desire for specific schools 
to match the range of occupations in GIS, 

GlS'a occupational file^ containing a sample of 1300 occupations 
from the D*0*T^ (which includes more than 20^000) was judged to be much 
leas complete. The liaisons- problra with completeness was not so much 
in terms of number j but rather^ in terms of why and how that particular 
set of 1300 wore included* Several classes of occupations were not 
included^ or emtshaaized enough (e.g*^ sports«related^ artistic occupa- 
tions); many specific occupations of interest to students were not 
included. On the other hand^ there was a great deal of redundancy 
(e#g*^ eleven draftsmen)^ an overemphaals on disappearing occupations^ 
and not enough stress on occupations in cMrging fielda* Furthermore^ 
according to the liaisons^ "there were too many requiring postgraduate 
degrees and too many at the lowest levels of emplopaent." Theae opinions 
of eKperienced teachers are supported by the data collected in the 
Initial Survey lof the 341 different occupations listed by students as 
their first choice for ftiture careers^ 41% were in the occupational file^ 
35%were related to occupations in the GIB file^ but 24% were not Included 
at all* 

Finally^ the Scholarship arid Financial Aid file^ conceded by TSC to 
be the weakest flle^ was judged Inadequate for the students in the five 
pilot high schoola* The need for financial aid information was great; 
that the scholarship file did not begin to satisfy this was evidenced^^in , 
the ^tent of use of that filei for all schoola in all periods combined^ 
less than 1% of the total uses were in the Scholarship and Financial Aid 
file^^althoughj initially^ a great deal of liaison tljne was spent in 
exploring this file. 

While use or itudent awareness is one index of the "goodness" of 
the information in GIS^ it is not an infallible indicator of importance. 
Slightly more than half (51%) of the total uses during the 15 months were 
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for collega information; threa«fourths of these were requests for infor- 
mation from the four«yaar college file. All schools made relatively 
little. use of the two-year college filei this low usage probably reflects 
student desire for a particular kind of information^ and not the quality 
of the information In this file* Overall^ 49% of the uses were for occu- 
pational information. These data support our hypothesis that while 
students verbalize needing more help with educational planning^ for very 
many this, means education-as-related-- to-occupational choice. 

As could be anticipated^ the two more acad^ically oriented schools 
.S5fi.S f ilfl. £84%. and 7^^.^^ respectively), 

than the other three schools^ and more use of the four-year college files 
(76% and 65%^ respectively) than the other schools. Schools A and for 
all periods combined^ requested information from the occupational file in 
69% and 62% of the Instances^ respectively. Proportion of use of the 
files differs from period to period as well as from school to school^ but 
not in response to the priorities imOE established. Furthermore^ on a 
schoolwlde basis^ proportion of use of a file is not directly related 
to the benefits to users on the Vocabulary or Decision-Making tests. Thus 

a An ideal set of educationally related data files should include 
information on two**yaar colleges | four-year colleges; selected 
post**baccalaureate progrms; and business^ trade^ technical and 
other TOcatlonal training schools, GIS's two college files were 
Judged to be fairly complete. 

m Students In New York City public high schools need more scholar- 
ship and financial aid Information than Is available in GIB, 

• An occupational flle^ which cannot feasibly contain a complete 
set of all occupational titles^ should smple occupations in 
relation to students' Interests and to educational objectives 
and expectations. The occupations included stmuld be non- 
duplicative^ and representative of the greatest possible number 
of diverse occupations. It seras to us that a reasonable 
sample should include large proportions of occupations with 
expanding or stable ^ployment opportunities^ and fewer with 
declining outlooks. Furthermore^ there should be a balance 
between occupations that students say they are interested In 
and occupations that "stretch" their Interesta 

• A set of files In an Ideal systan should be better inter- 
related than are the GIS files. For exanple^ training places 
for an occupation should be available In (at least one of) the 
"schooling flies," 

m School variations in the extent of use of the GIS files Is more 
related to in-school needs than to externally Imposed prioritieap 

• The proportions of the schools' uses of a file do not seem to 
be related to the outcomes on the impact test measures. 
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The data i^ae not analyzed In tems of the individual student *s 
eKparienca with one or another of the major files. 

* * * 

Completeness can refer to the nimber and kinds of categories on 
which colleges and careers are classified and on the range of character- 
is tics within the categories* The college files contain more categories 
and characteristics than does the occupational file^ 25 categories and 
600 characteristics In the four-year college file; 21 categories and 350 
characteristics in the two-year college file; and 7 (selector) categories 
and about 80 (selector) c^ occupational flTje 7 Th 

difference means that the user has less choice of input (and lass infor- 
mation in output) in the occupational file than in the college files. 

The liaisons rated the categories and characteristics highly complete 
in the college files. They suggested Inclusion of other categories (e*g*^ 
size of college departments^ and high school course prerequisites for 
college majors). The occupational file was not rated highly either for 
completeness of categories and characteristics^ nor for clarity or logic- 
alness. The liaisons suggested adding a category to the occupational file 
describing the educational courses or majors which relate to the occupa- 
tions. IWDOE's analysis of student use of the available categories and 
characteristics revealed that certain categories are used much more 
frequently than others. Frequency of use relates to at least three fac- 
tors* importance and relevance to the student; students' familiarity 
with the topic; and the quality of TSC's treament of the category. 

Xn the four-year college file^ the categories of Majors and Location 
were OTployed in 98% and 81%^ respectively^ of a sample of student 
searches. Coeducation and Gompetitlveness were used in about tw-thirds 
of the searches eKamined in February 1974. The next two categories in 
rank order were Sizm of Total EnrolLffient and Costs^ both of which were 
used far less frequently (24% and 23%, respectively). There were 11 
categories which were raployed in 5% or fewer of the searches examined. 

The use of two-year college categories differed slightly. Fewer of 
the available categories were used^ and half were employed In 5% or fewer 
of the searches. Most frequently used were Curriculittns Leading to Asso- 
ciate Degrees and Location. 

Despite the fact that there were so many fewer (7) categories avail- 
able for an occupational search^ two categories were used in fewer than 
9% of the searches* The ronalning five were: Cluster (97%)^ followed by 
Interests (83%)j Levels of Occupations (82%), Levels of Formal Education 
(73%)^ and Aptitudes(55%), 

These frequency of use findings parallel the results on the Initial 
Survey (conducted one cronth after the frequency of use analysis) in which 
students indicated that majors offered^ location^ and tuition were of most 
importance in choosing a college. Similarly^ in deciding on a future 
career^ all respondents could nrae their first choice with varying speci» 
fiuity^ and indicated that that choice was a function of their own 
interests and abilities. 
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Early in 1975 (January-February), itudents in three schools who had 
used the computer ware compared with non^users In the same classes, on 
four instriments measuring the mount and variety of information they had 
about colleges and careers. On each and every one of these open-ended 
measures, uiers consistently produced rare information and more varied 
information than non^^userss Atoost without eKceptlon, the content dif- 
ferences in mean number of statCTients and mean UOTiber of categories (in 
favor of the users) was with respect to information in GIS* That is, the 
differences between users (many of whom had indirect CAG experience, and 
eKperience that predated the test administration by 6 months to a full 
yaar) and non-usera were almost entirely in content available in GIS. More- 
over^ a highly statistically significant^ d 

eKisted between users and non-users | the Vocabulary itans were selected 
from the GIS Student Study Guide* 

While it is quite clear that the GAG experience improved users' 
college* and career»related vocabulary, and their descriptive and organic 
national ability with regard to college- and career-related information, 
only about two-»thirds to three-quarters of what students think is important 
about colleges and occupations are Included in the Guidance Information 
System. Students want mre specific infomation about college major fields 
of study- "including the range of available courses^ an estimate of a depart 
ment-s reputation vis-a-vis other colleges, a description of the colleges' 
educational style in terms of lectures, seminars. Independent studies, and 
so on* Students consider some information (not in GIS) about occupations 
important as well; they are interested in knowing more specifically abouu 
the content of the training (i^e., necessary courses) in addition to where 
that training can be obtaineds They are also interested in information 
about occupations in general that is available only in GIS as a descriptor 
about a particular occupation. And, they think it ijaportant to know about 
the life styles that are associated with occupations * In conclusioni 

m The classification of college Information (categories and charac- 
teristics) in GIS is fairly complete from the liaisons* point 
of view, but contains only about two-thirds to three-fourths of 
what students feel is important* 

• The classification of occupational infomation in GIS is not as 
complete nor as extensive as the college files , Again, GIS 
contains only about wo--thlrds to three-fourths of what students 
want to know about occupations. 

m Certain categories are used in searches conducted by students 
more frequently than other categories. In the occupational file, 
this is due to the limited availability of categories, and the 
ambi^ity of at least ^o* In the college files, the most used 
categories reflected students' statOTents of what was most im- 
portant to then. It rraiains for future research to determine 
whether the availability of clearer and more complete categories 
and files of information would alter what students ask about^ — 
i*e,, lead to eKpanded inquiries. 

• Students with CAG eKperience have more and more varied informa- 
tion about both college and careers than students who were not 
exposed to CAG. 
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0 MiGh of the difference between users and non -users In amount 
and variety of infomation about colleges and careers reflects 
Information available in GIS. Furthermore^ In the test of 
meaning of ItCTS (TOcabulary) selected from the Guidance 
Information Syst™, users outperformed non-users* No data is 
available on the benefits to an individual who used any set of 
categories or characteristics in any file. 

m A computerized approarfi to providing college and career in- 
formation Is a valuable approach for building or improving a 
career Information progran at the high school level* 

m An Ideal aystm should Include categories of information stu- 
dents and professionals agree are Important^ but many of the 
less frequently used categories^ or ones not usually considered^ 
may not be irrelevant. The Infomation they contain may be of 
a kind that students do not thlric to ask about at first* It 
may be Important to retain some of these categories since 
expanded choice may result from their availability In a system^ 
at least by some students who become aware of their importance. 

m To clarify the reasons for little or no use of certain cate'* 
gories^ further study is needed to determine which may be 
truly Irrelevant^ which are little used because they are 
difficult or confusing In the GIS form^ and which are impor- 
tant but not often thought about. 

* * * 

Other analyses of the GIS sofTOare components that were conducted 
reveal that ^ch of the liaisons' day-to-day criticisms concerning ac- 
curacy of the information and recency of information had some foundation. 
Most of these have ijapllcatlons for data collection and update procedures 
and will not be discussed In this chapter. 

As a siffiEiary statfflient about the Guidance Information SystOT^ we 
conclude that there are more problans with software- than with hardware- 
related aspects of a cmnputerl^ed college and career information retrieval 
ayst^* Nonetheless^ GIS Is an effective system and( although not easily 
used by students alone without staff support^ produces many benefits to 
swdents^ staff^ and the schools Involved. We now turn to a sumary 
section on these benefits— the direct benefits as reflected in college 
applications data^ In vocabulary^ decisionmaking, and the amount and 
variety of Information obtained by student GAG users. 



The mmber and types of colleges students apply to Is more a function 
of costs and Income, Including scholarships and financial aid, than it 
appears related to GAG use* In this demonstration we could not directly 
compare applications of users with non-users; the closest approKimation 
we could reach was a comparison of the applications of the June 1974 
graduates with those of the June 1975 graduates* Better tests of the 
effects of GAG on college applications and on rates of acceptances and 
rejections are possible and might be considered in the future* 
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We looked at CUNY applieations^ SUNY applications^ and applications 
to all "other" 'colleges. Combining all applications for all schooli^ 
there was an overall decrease in the average number of applications per 
graduata (APGe) in 1975^ as compared with 1974^ from 1.5 to 1.2. The 
total nimber of different colleges applied to also decreased^ from 385 to 
331, All schools except A and D showed a decrease in APGs^ which was most 
dramatic at School C- 

In 1974^ 63% of all graduates applied to CUNY; in 1975^ 54% applied. 
The SUNY APGs decreased from .41 to .31 while the other college APGs went 
firc^— te47- tO" *37v - There wAs^ = towawSj^^=^ Incraase .In. the. total .^nmitoer. .of . 
different SUNY colleges applied to. For both yearsj 90% or irore of all 
applications were to four-year colleges* The individual schools main- 
tained their rank order in both years i generally School C filed the moat 
APGs followed by Schools D and E; Schools A and B were at the bottom. 

Looking at the acceptances of applications with known outcomes^ there 
was a slight increase in the percentage of non-CUNY acceptances in 1975; 
this was ahnost entirely attributable to a much higher rate of acceptanca 
to SUNY colleges* The non-CUNY rate of acceptance was 75% in 1974 and 
79% in 1975; the SUNY acceptance rate increased from 73% to 81%, 

• The effects of CAG on college applications and acceptances are 
not clear from these analyses* Liaisons report many instances 
in which students located colleges through GIS and were ac- 
cepted by than; the staff also reported that CAG frequently 
had the effect of modifying students^ college choices (either 
expanding or constricting thm)^ to better accord with reality. 
Obviously^ better and more research is needed for system- 
atically isolating the effects^ if any^ of CAG on collega 
declsion-making and choice, 

• The overall decrease in APGs might be attributable to the fact 
that CAG may have brought about fewer^ but more realistic^ 
applications. This is supported by the increase in rate of 
acceptance to SUNY colleges. These findings^ however, may 
have to do more with general economic conditions than with 
school-, student-, or CAG^related causes* 

9 The decrease in CUNY applications reflects a difference in CUNY's 
processing and differences In IRDOE-s data collectlen In 1975* 

• About 90% of all applications filed in both years was to four- 
year colleges* This agrees with students' preferences for 
this type of education, and their low use of the two-year 
college file* It does not necessarily mean that a two-year 
file is not important, nor that the GIS file is inadequate* 

m There is some slight evidence to suggest that the particular 
high school a student attends greatly Influences the colleges 
to which he will apply, 

* * 
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The findings with respect to the user/non-user differences on six 
measures of the direct impact of GAG were more clearcut^ and have been 
referred to throughout this chapter. 

The preference for the computer as a source of information about 
colleges or careers was in the middle of a list of sources* Overall^ some- 
what more users (36%) than non^users (28%) preferred the computer for uol*- 
lege information^ although it ranked below the college of f ice-counaelor^ 
visits to colleges^ and catalogs ^ Individual schools differed* School E's 
non*»uieri preferred the COTiputer more than users at other schools; at 
Schools D and C fewer users than non^^users preferred the computer « In 
tsrmls^of tareets^ prefeteMfe for the career of £tce*co^ 
for all schools combined^ followed by talking to people in different 
careers^ and visiting work sites « Both users and non-users preferred 
the computer less for carear than for college Inforaationj although users 
ranked it higher than non^users. The individual school differencei were 
very similar to those obtained for the computer as a college informational 
source . 

Preference for the computer as an informational source is not related 
to the benefits students get from the GAG eKperlence in terms of cognitive 
meaaures. School 1 students were most enthusiastic about the computer^ but 
in this ^eteol users differed least from non-users on Decision-Making; 
Sdiool C USUI'S w£^'e not ardent fans^ but they did d^onstrate superiority 
over non-*uaer i an Vocebulary and Decislon-*Maklng. 

IlIw ^jrirf^rence between users and non-users on Vocabulary was statis- 
tieatl^y iSv^lxlcant in favor of users- The individual school differences 
all tovorM usersj but the variations in schools and in CAG method gave 
no ^ii^^ilflc\ gently greater advantage to users* The data suggest that a 
school's performance on this measure is related to the general ability 
level of students. 

The Declslon^Haklng results are stoilari users performed significantly 
better than non*u isr?;. on this measure^ Although the same direction of 
benefits held lov aqrh individual schoolj there was no statistically sig- 
nificant differeritia? advantage to users. There was^ however^ a tendency 
for the users in schtmls of lesser acadOTic level to show greater benefits^ 
possibly because of the low test ceiling. With this instrument^ there 
BB&ms to be some posi::lve relationship between the magnitude of advantage 
to users and the pronartion of users in the sctool who conducted searches. 

On the Self and Ideal College Questionnaires^ users produced wre In- 
formation and mora varied Information than non-users; users employed a 
greater n^ber of categories which were In the Guidance Information System* 
Moreover^ about 30% of the categories generated by users (Self College) 
were e^e /usive to th^i-ME^ere not OTployed by non^usera. 

On th^ Self and Ideal Occupation Questlonnalrea (which appear to tap 
different student eoncerns)^ the mean differences in various scores (number 
f^f st^ri Atts^ number of categories^ nimber of unique categories^ cate- 
gories rejitad to GIS^ and personal- vs. Job-oriented categories) clearly 
favor*4 <Sd a^ers. The users mployed more and a greater variety of infor- 
mation Aa ' more GlS-related Infcimiation, Users^ moreover^ tended to be 



123 



more concerned about the educational or training requirements of an occu- 
patian and aemed to be more attentive to differences in the wording of 
the questions on the instruments* 

On all four instruments^ approximately three« four the of the information 
users and non-^users thought it was Important to know about was included 
in GIS; some of the information not in CIS might be considered in the re- 
design of the data files. To sunmarlze: 

m The computer as a source of college and career information ranks 
. ..... i^ .^h§_miM It was preferred by^ students 

more for college than for career information^ reflecting all our 
other data describing this file as less adequate than the college 
files. 

m A larger proportion of users^ overall^ than non-users preferred 
the computer as an Information source. There were variations 
airong the individual schools; although the computer was ranked 
low in Schools C and non-users tended to rank the computer 
higher than users^ indicating that these schools may have 
better sources for information and that users are better able 
to make that Judgnent- 

9 The students in School non-users and users^ rank the com- 
puter highly as a source of information. Their preference for 
the computer does not influence the results on vocabulary or 
decision-making I on the latter measure In particular^ user 
benefits accrue to schools with a great proportion of students 
who used CAG's search capability* 

• The data on source preferences seem to suggest that students^ 
especially users^ do not differentiate clearly between "counselor" 
and the computer -as sis ted counseling eKperience. This question 
might be the basis for future research activity. 

• Users performed significantly better than non-users on measures 
of Vocabulary and Decision-Making. The findings seem to in- 
dicate that students poorer In decision-making ability might ^ 
benefit most frOTi the GAG experience-^especially from a file 
search* 

m Users have more infomatton and a wider variety of Information 
about colleges and careers than do non-users. The advantage to 
users is attributable to GAG, since they employed a greater 
variety of GIS-related categories. 

m Two-thirds to three-quarters of the Infomatlon students con- 
eider taportant about colleges and careers ig in the TSC system. 
The ranainder of the Information Is not. 

* * i€ 

On the basis of all our experiences and findings we reconmend that 
serious consideration be given to the expansion of a computer cassis ted 
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guidance project. Such an expanded progrmi should encompass or build on 
the components that have been tried successfully by the five pilot high 
schools. 

Specifically^ we recpmiend that a hardware confl^ration similar to 
TSC's be used* The teletypewriter teminal should be retained since it 
was indestructible and permitted the generation of hard copy with multiple 
carbons* There are more sophisticated models that are faster and less 
noisy, but in our Judpaent the noise and speed of the KSR-33 attracts 
students, and staff quickly adapts. In schools with students of lesser aca- 
demic abllit^^^ thffli to fp llow the outpu t J. a yariable .. 
speed model might be considered for high schools whose students are reading 
at or above grade level. 

We reconroend that schools be assigned time in advance; the equivalent 
of one day a week access per school is adequate* MaKimal use will be 
attained if schools are scheduled half -days or smaller portions of days 
rather than whole days, A "buddy" systra of pairing 3 to 5 schools would 
eliminate the need for external managCTaent, The schools in each buddy- 
group could arrange their own trades for special events, and v n^td^ If 
one experienced a breakdown, call a buddy to see if one of tb^ n > ild 
use the time, 

A staff person in each school needs some ttoe allotted for noordi- 
nating responsibility; half-time is suggested* This can, of course, be 
divided fflnong different staff and every effort should be made to increase 
the number of in-school staff who are familiar with the operation^ VJith 
a revised systm, there should be less staff time needed to operate the 
terminal — with more time devoted to orienting students and following them 
up in an advising relationship* in time, students will learn of the 
availability of the system in their schools; witn system improvements, more 
students should be able to access the Information with considerably less 
staff help* 

Except for research purposes, there seras to be little reason to eg- 
tabltsh eKternal priorities for all schools for use by students in a par- 
ticular grade level or at a particular time of year. The high schools are 
cognizant of their own needs and will adapt the systCTi--provided, of course, 
it has built-in f leKibllity--to meet these needs* Schools x^ll also estab- 
lish their own patterns of use* We recoimend^ however, that direct inter- 
action and the search mode be encouraged* Althou^ the data with respect 
to direct interaction effects on students is not definite, there are other 
serendipitous benef its--lncluding increased involvement with staff and 
increased flow of students to the area (office and personnel) in which the 
terminal is located* With CAG^ students seek out help and advice more 
frequently and earlier in their high school career. 

Searches are recommended because it is the only way students are 
exposed to (GlS's) decision-making logic* The data suggests that improved 
decision-making ability Is associated with searches. 

Without modification, GIS might be eKpanded most feasibly and use- 
fully to schools of lesser acadanic ability, to very large schools, and/or 
to schools with less well-developed college orientation. However, GIS can 
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become the baeis for building or impro^dng a school* s college office or 
; career education thrust -^^provided^ in the latter instance^ that super- 
visory staff ba assigned this function primarily* A modified system^ with 
an improved occupational file^ would be more adaptable for this purpose* 

Whether a future systCTi sbDuld build on GIS^ or whether a new system 
with new data bases should be developed depends partly on funding. We 
will not deal with this issue here; instead^ we will attempt to describe 
the ideal systm that is more responsive to the needs of New York City 
high ichool students. 

An ideal COTputerized systan should provide much simplified access so 
that greater numbers of students could process their own requests for in- 
formation. Similarly to GIS^ a user should be able either to conduct a 
search or ask for a description of an Item. An ideal system should retain 
GIS's narrowing--inverted pyramiu--* logic or a similar program structure. 
All fllea^ however^ should be similarly constructed (e.g.^ no division 
between selectors and deicrlptors as in the GIS occupational flle)^ and 
should be cross-referenced: for every occupation included^ there should 
be apprbprlate places for training. 

The number of files needs to be Increased to include vocational and 
business and trade schools and a working scholarship bank^ at a mlnlminn. 
Within each flle^ the Information might be coded to meet the needs of 
special interest groups or subpopulatlons (e,g*^ the handicapped and the 
incarcerated). Great care should be eKercised in developing and applying 
rigorous standards for inclusion In a permanent data file. The same 
standards that apply to two-year and four-year colleges should be applied 
to alternative schools. And we urge that In each and every educational 
file students be apprised that a degree from a not*yet-accredlted institu- 
tion may not have the general acceptability as does a degree from an 
accredited one. 

There are some reconiEie. ad changes that need to be considered in 
files of four^yesr and two-year colleges. The findings suggest that some 
GIS college categories and characteristics should be eliminated or 
shortened^ while others should be added. As an offshoot of this demon* 
atration study these data are currently available* 

Changes in an occupational file will be more complex because of the 
necessity to sample occupations. It seems to us that the smpling should 
reflect the best opinions of professional educators--as well as an educa- 
tional philosophy^-and the state of the real world. We have already 
Implied that^ for exmple^ more ©nphasis should be placed on occupations 
with stable or expanding opportunities and fewer occupations that are 
becoming obsolete. The file should also Include occupations that stretch 
students ' horizons and occupations most frequently asked about. They 
ihduld be non-duplicative and representative of the greatest number of 
diverse careers; moreover^ to ^he extent possible occupations that ^form 
career ladders should be Included, 

Most of the occupations should require a minimum of a high school 
education if the system is directed at secondary level students; some low 
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skill level occupations should be retained^ however^ to illustrate the 
"yalua"o£ education. Certainly^ even for New York City students^ the 
file Bhonld include occupations other than urban ones; but in what pro- 
portions? Simllarlyj there should be occupations at the lower and inter- 
mediate levela and at the college^ postgraduate^ and doctoral levels; 
these should possibly reflect their representation In the real world of 
work. 

Within the occupational file^ categories and characteristics need 
to be clarified and expanded. The data on which to base some modifications 
are available. To determine which of the suggestions are worthy of future 
adoption^ there needs to be some consideration of what the use of an oc- 
cupational file (or for that matter^ what the use of college files) Is 
intended to accomplish* Once there is some consensus^ then the guidelines 
for selecting and smpllng^ expanding or restricting will follow. 
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POSTSCRIPT 



The astounding den»nstratlon of CAG's benefits to student usera^ 
together with urging of the school administrators and staff prompted the 
Board of Education's Division of High Schools and First National City 
Barit to continue the project in the sme five schools for the 1975-76 
acadaalc year. This was consldared an Interim measure--a "holding pattern*'-- 
which would keep CAG operational at a mintaal level so that decisions could 
be made with respect to future expansion of GIS and/or modification. 



GAG. 1975-1976 

Mpl^entation in 1975-76 was considerably aggravated by New York 
City's budget crisis which resulted in cutbacks in personnel^ including 
manpower shortages in the public schools. Due to the severity of the fis- 
cal problems^ the Division o£ High Schools was forced to reduce Its 
contribution to the CAG project by half ^ from one full teaching unit al- 
lotment per school to one-half unit* They continued absorbing all 
telephone costs. Considering the extent of the Division's across-the- 
board cutbacks that year^ the allotment of any proportion of a teaching 
unit indicates substantial interest and support. 

Citibank's funding was also decreased^ affecting only iBDOE's re- 
sponsibility in 1975^76; we were to serve only in a managerial and 
supervisory capacity^ ""developing and maintaining calendars and schedules^ 
ordering CAG-related supplies^ training staff^ and so on. Although IMOE 
staff time was Insufficient for collecting elaborate data^ and its con- 
tractual obligations did not Include this type of activity^ some data on 
time and uses ware kept, 

A one-week UFT strike in Septaaber 1976 Increased the usual start-up 
problems* There were other delays as well; because of the teachers' 
contract^ School B which had intended to retain the guidance intern (who 
served as the full-time liaison in periods 1^ 2^ and 3) on a reduced time 
schedule could not do so. When school finally opened in September^ School 
B had no liaison. A teacher of business education^ interested in the 
project^ was selected by the principal and was trained by IMOE, The 
liaisons at the other four schools ranained the same^ although their other 
in-school responsibilities had to be adjusted to compensate for the 
reduced allotaent. 



During the 1975*76 school year^ which corresponds with Periods 2 
plus 3^ the computer was used for a total of 464 hours. This Is a 13% 
decrease from Periods 2 and 3. Considering that staff time decreased by 
50%^ the decline in number of hours is significantly less than was eK= 
pec ted. 
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Moreover, and perhaps of mora Importance, was the ieparate schools' 
pattern- Sctool A used about 111 hours in 1975-76 (115 in Periods 2+3); 
School B used 118 hours (an increase from the 89 hours used in Periods 
2+3); and School E used 166 hours in 1975-^76 (as compared with 191 hours 
in the previous year), a 137o decrease. The overall decrease is mostly 
attributable to Schools C and D which used about 15 and 53 hours^ respec-^ 
tively, in 1975^76 (as compared with about 29 and 110 hours in Periods 
2+3) i the tljae used by these two schools decreased by about 50%, 

Similar results were found in exmlhing the nmnber of student usee 
of the computer* In 1975^76^ the computer was used a total of 2953 times-- 
a 15% decline frOTi the 3457 uses in Periods 2+3, Againj the decrease was 
far frfflm consistent from school-to-schools Schools A and B increased their 
usage by 3% and 13%^ respectively. School E evidenced a 26% decrease^ 
while Schools C and D used the computer much less--54% and 59%^ respective- 
ly, Thus^ while Schools and E decreased usage^ only in C and D was 
the decrease more than would be predicted from the reduction in staff time. 

With the ^ceptlon of School C^where after Period I interest in GAG 
had been moderate at best^ enthusiasm at the other four schools remained 
high* There was a sense of disappointment about the reduced time^ and an 
intense feeling of curiosity about CAG-s future. The School B liaison^ 
who was n^ to the project^ was continually TOa^ed about the kind and 
quality of service she was able to give students through GAG, The liaison 
at School A expressed her school's eagerness to see initiated the system im 
provements that she and other liaisons had so diligently suggested. 



Proiect "ACCESS" 

During the 1975-76 school year^ IRDOEj Citibank^ md the Division of 
High Schools explored the future of the project. Representatives frOTi 
many other agencies were called in as ^re compleK and detailed problons 
arose* These people rmt with their own staffs and advice^ suggestions^ and 
special-interest concerns were brought back for discussion- 
Meetings between IRDOE and Citibank staff started in June 1975 and 
actually contiTOed through August 1976- The first round focused on 
questions of "need": Is there a need for additional data files (e,g,^ 
vocational schools and scholarships and financial aid)^ and is there a 
need to extensively modify GIS's four-year jollege^ two-year college^ and 
occupational files? And^ can the need to continue the service better be 
met by building on TSC's Guidance Information System or by starting to 
design a system designed for and owned by New York City? 

lOTOE^ with the cooperation of Citibatdc^ prepared a proposal for 
Project AGCTSS (Automated Career^ College^ Educational^ and S^cholarship 
£earch)^ targeted to the needs of the New York City public high school 
studente.and built on a complete and educationally sound data base* The 
major part of the proposal described key parameters of a computerized 
system including the validity of records^ relevance of the taKonomy^ and 
comprehensiveness of the subsyst^s. Several possible subays terns were 
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descr±bad--leie*than-baccalaureatej four-year college^ financial aid^ and 
QBwmmvBm Data file alternatives^ progrm logic^ and hardware and soft- 
ware systOT queations were eKralned* Althou^ this proposal was budgeted 
in terms of developing a systOT that was designed to be the property^ and 
responsibility of the Board of Education of the City of New York and 
Cltlbardi^ its specifics were adaptable to modifying already existing 
progrms* 

,_The second round of negotiations involved systOTS and computer ex- 
perts from Citibank^ the Office of the Deputy Chancellor^ the Bureau of 
Management Information and Data Processing^ the Graduate School of the 
City University of New York^ and IBM, These discussions .took place during 
October -December 1975^ and were concerned with hardware problems^ com- 
munications^ and on-going managerial (technical and educational update) 
issues. Differences of opinion mong systems experts concerning whether 
or not a computerlEed information retrieval system could work at all^ or 
could operate on the Board of Education's hardware flourished* Estimates 
of what it would cost to "put up" such a retrieval progron on the Board -s 
hardware ranged from a conservative $10jOOO to a record high of a quarter 
of a million dollars. And this^ of course^ meant only adapting or up- 
grading the hardware^ and did not Include costs relating to the software 
package^ the building of a data base^ nor to on-going operational costs 
for 100 public high schools. 

Interto measures were considered # The Graduate School offered free 
computer service to 10 to 20 high schools and use of their computers for 
research (i*e*^ file construction) purposes. II©OE^ the Graduate School^ 
and IBM located several appropriate software packages^ Including an 
Informational retrieval package that would have been available to the 
Board of Education at a nominal charge^ since other city agencies had a 
licensing agreement with the developer. Other activities designed to 
move computer^asslsted ^Idance into an eKpanded nraber of high schools 
included proposals to lease GIS^ or to purchase TSC's mlni-computer+GIS. 
IRDOE revised its plan so that data, collection could be accomplished in 
smaller^ less costly stages* Although Citibank favored one-source funding 
of a GAG project j a great amount of time was spent in considarlng the 
question of cosponsors. 

It was at this point that a more basic stumbling block CTierged--and 
one that continues to be the major obstruction. While both Citibank and 
the Division of High Schools were in favor of an eKpanslon^ Citibank 
needed a comiiteient from the Board of Education that it would Implement 
and man a new systan--either Project ACCESS or a much modified GIS. The 
Board^ like all other agencies in New York Clty^ budgets on a year-to- 
year basis. General revenues are divided mong all city agencies^ and 
economic conditions affect the budget allocations. Moreover^ there was 
some question as to whether it was legal-^lndlng--or appropriate 
for the Board to coranlt (earmark) funds In advance. Although very desir- 
ous of the continuation (and expansion to 10 high schools) ---as evidenced 
by the Board's allocation of teaching units for the 1976^77 school year-- 
Gltibank needed assurance qf a longer-range coMiltment to a project that 
would require so large an Investment and had to vote it down. The last 12 
students used the computer on June 22^ 1976. 
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Fair too frequently dCTionstrations conducted in school settings do not 
provide policy makers with sufficiently convincing data on which to base 
future decisions. More often than not^ these evaluations report that 
although staff and students like the daaonstration progrm^ there were no 
meaiurable student gains. Such conclusions leave the reviewer without 
clear directions^ Implying perhaps that another year Is warranted in which 
to improve toplementation strategies or measurment techniques* 

The results we reported as accniing from the CAG demonstration are 
clear and definitive. Despite the fact that a computer-assisted guidance 
project may be more useful in soto schools than in others^ all the evi- 
dence taken together indicates unequivocally that benefits accrue to 
students and to schools* 

As professionals in educational researcli*and development^ we are some- 
what saddened by the fact that CAG^ like some other special-funded demon- 
strations^ is "all dressed up with no place to go*" Alternative suggestions 
for dealing with similar circus tances in the future may lie^ in part^ in 
long-range funding of essential City services at a minimal level^ or in the 
initial planning of the project* In the latter instance^ some provision 
could be made by all participants (funding agency^ grantee agency^ and 

site institution) to establish standards relating to cost and value which 

if met by the pro ject- -would lead to its consideration as part of an 
on-going operation. 
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TABLE Al 



NmBER OF JUNIORS AND SENIORS TAKING INITIAL SURVEY OF STUDENT 
ACTIVITIES AW PLANS IN MARCH 1974^ WITH PERCENTAGE WTO HAD 
PRIOR CAG EXPERIENCE^ BY SCHOOL 





Grade 11 


Grade 12 




School 


N 

Juniors 


% With 
Prior CAG 


N 

Seniors 


% With 
Prior CAG 


Total 


A 


236 


16% 


198 


6% . . 


434 


B 


246 


13% 


90 


0 


336 


C 


187 


25% 


198 


0 


385 


D 


201 


40% 


182 


3% 


383 


E 


193 


33% 


158 


6% 


351 


All 
Schools 


1063 


25% 


826 


3% 


1889 



*BecauBe encire classes were teatad^ soma lOtli grLidurB took the 
Initial Survey* TliGlr responsGs wero tioL Lnclutiud in thy analyse 
Prior eKporience with CAG was detennined by examination o£ User 
Sign-In records^ which gave the studeiid's nimie^ grade^ and date o 
use « 
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TABLE A2 

DETAILS OF ADMINISTRATION OF INSTRUMENTS FOR ASSESSING mPACT 
OF GAG ON STUDENTS, JANUARY-FEBRUARY 1975 



A. NUOTER OF STUDENTS TESTED ON EACH INSTRUMENT, BY SCHOOL 



Ins Crumcn t 


School 


All 


A 


B 


rif 


U 




Schools 


1. Preferred Sources of 
CollBge Btid Career 
Information^ 


77 


86 


88 


68 


86 


405 


2. Vocabulary 


7^ 




92 


86 


81 


413 


3, Self College 








48 




48 


4, Ideal College 




51 




56 


55 


162 


Closed-Ended College 








54 




54 


5.. Self Occupation 




55 






58 


113 


6* Ideal Occupation 




60 






57 


117 


7* Decision-Making 


84 


87 


92 


69 


87 


419 


Date of Testing^ 1975 


1/20 


2/18 


2/21 


1/10 


2/28 




Grade Level 


12 


12 


12 


12 


11, 84% 
12,16% 





*Tv?o questions placed at end of Decision-Making iListrument; the analysis 
used only respondents giving three choices* 

^^Data not presented in this report; the multiple-choice format precluded 
obtaining qualitative Information^ and user /non-user differences were 
slight, 

^^School C is the only one where the same students took both Vocabulary 
and Decision^Making* 



B* TOTAL NUMER OF DIFFERENT STUDENTS TESTED^ NUMBEIR OF CAG 

USERS AND TON-USERS^ AND TOTE MD MCENCY OF GAG EXPERIENCE^ 

BY SCHOOL 





N Students Tested 


% Who 




For 


Users, 


Percentaga By: 


School 






Non- 


Were 




Type of CAG 


Time of Most 


Recent Use 




Total 


Users 


Users 


Users 




Experli 


ance* 


Period 1 


Pd. 2+ 9/74 














Indirect 


Direct 


(2/74-6/74) 


to Test Date 


A 


159 


94 


65 


59% 




48% 


52% 


57% 


43% 


B 


166 


76 


90 


46% 




42 


58 


66 


34 


C 


92 


33 


59 


36% 




64 


30 


76 


24 


D 


155 


81 


74 


52% 




68 


28 


86 


14 


E 

* All" " 
Schools 


_168_ 
740 


105 
389 


63 
351 


62% 
53% 




48 

52% 


48 

45% 


23 
57% 


77 
43% 



*Where Direct and Indirect do not equal 100%^ the remainder did not indi- 
cate the type of experience; for all schools^ these us^ totalled 3%. 



146 



135 



TABLE A3 

mM SCORES ON VOC^ULARY AW DECISION-MAKING FOR HDNDRS AOT 
NDNHDNORS SUBGROUPS^ USERS AMD NON-USERS^ BY SCHOOL 



A. VDCABULMY (MAXBflJM SCORE ^ 12) 







Users 


Hon-Users 








School 


Honors 


Non- 
honors 


Honors 
Advan- 
tage 


Honors 


Non- 
honors 


Honors 
Advan- 
tage 


All 
Honors 


Non- 
honors 


Adyan- 
taRB 


A 


N 
M 


38 
6.7 


10 
' 5.4 


+1.3 


11 
5.8 


16 
4.2 


+1.6 


49 
6.5 


26 
4.7 


+1.3 


B 


N 
M 


0 


35 
6.5 




0 


44 
6.6 




0 


79 
6.6 




C 


N 
M 


0 


33 
8.2 




0 


59 
7.0 




0 


92 
7.5 




D 


N 

M 


3 

10.0 


40 
8.2 


+1.8 


10 
8.2 


33 
8.0 


+0.2 


13 
8.6 


73 
8.2 


+0.4 


E N 
M 

All 
Schools 
N 
M 


17 

7,8 

58 
7.2 


38 
7.7_ 

156 
7.5 


_+0.1_ 
-0.3 


9 

30 
7.1 


17 
_7._2 _ 

169 
6.9 


+0.1 
+0.2 


26 
7,7 

88 
7.1 


55 
7.5_ 

325 
7.2 


+0_. 2_ _ 
-0.1 




B. DECISION-MAKING 


(MMIM 


UM SCORE 


« 6) 




A 


N 
M 


36 
3.8 


10 

3.2 


+0.6 


17 
2.8 


21 
1.7 


+1.1 


53 
3.4 


31 
2.2 


+1.2 


B 


N 
M 


d 


41 

3.8 




0 


46 
3.1 




0 


87 
3.4 




C 


N 
M 


0 


33 
4.4 




0 


59 
3.6 




0 


92 
3.9 




D 


N 
M 


14 
4.8 


24 
4.0 


+0.8 


2 

2.0 


29 
3.6 


-1.6 


16 
4.4 


53 
3.8 


+0.6 


E N 
M 

"All 
Schools 
N 
M 


13 
J.J _ 

63 
4.3 


37 
3 .J _ 

145 
3.9 


+1.^ 
+0.4 


16 
_ 4.8_ 

35 

3.6 


21 
_3.3_ 

176 

3.2 


+0.4 


29 
5.0_ 

98 

4.1 


58 
_3.J 

321 
3.5 


+1.4 
40.6 
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TABLE A4 

STUDENTS' PLANS FOIJ£)WraG HIGH SCHOOL GRADUATION, BY SCHOOL 
Aro GRADE (RESULTS IN PERCENTAGES)* 





School 


All 


Plans 


A 


B 


C 


D 


1 


Schools 




11 12 


11 12 


11 


12 


11 


12 


11 


12 


11 


12 


Four^Year College 


30% 22% 


19% 


13% 


55" 


4 38% 


40% 


50% 


23? 


i 24% 


32° 


i 31% 


Two-' Year College 


3 


12 


5 


11 


5 


10 


6 


8 


5 


8 


5 


10 


Special Training 
(e.g-^Trade School 


/ 




9 


8 


5 


4 


2 


4 


6 


4 


6 


4 


Job and School 


42 


41 


34 


34 


24 




37 


26 


38 


44 


35 


36 


Job €e School Later 


7 


10 


8 


7 


5 


5 


7 


2 


8 


8 


7 


6 


Get a Job 


4 


6 


16 


21 


4 


8 


3 


5 


13 


7 


8 


8 


Military 


3 


2 


5 


1 


1 


1 


0 


1 


3 


1 


3 


1 


Other 


2 


1 


4 


2 


1 


0 


4 


-> 


3 


3 


3 


2 


No Response 


2 


4 


0 


3 


0 


0 


1 


1 


1 


1 


1 


2 


Total % Planning 
Further Schooling 


89 


87 


75 


73 


94 


91 


92 


90 


80 


88 


85 


87 


Total % Planning 
Work Bl Schooling 


49 


51 


42 


41 


29 


39 


44 


28 


46 


52 


42 42 


N: 


236 


198 


246 


90 


187 


198 


201 182 


193 


158 


1063 


826 



^From Initial Survey of Student Activltlei and Plani^ March 1974. 
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HABIM A5 

mm LIAISON RATINGS OF THE ADEC^ACY OF THE GIS COLLEGE FIUS 
(Categories Listed In Order of Rating of Frequency of Use) 





N Charac^ 






Variables 


Rated 








teristlci 




Cocip lete— 
ness 


Accu- 
racy 


Clar- 
ity 


ADDroofi — 

r' ^ ^ 

ateness 


vance 


Fre- 


Category 








quency 

\J A. U 3 € 


Acad^lc Progrftflis of 
Study Clfajors") 


^4 / 


1 Oft 




2.6 


2.7 


2,4 


2.6 


2,8 


3,0 


Geographic Location 


Dj 






2.9 


3.0 


3.0 


2.6 


2.9 


2.8 


Flnaneial Aid 








2,8 


2.7 


2.9 


3.0 


3.0 


2.5 


Compe 1 1 1 i vane a s* 


b 






2.6 


2.5 


1.9 


2.2 


2.1 


2.4 


CoedueatloQ 


5 


5 




3,0 


3.0 


2.4 


2.3 


2,3 


2.3 


Control (Pub, vs* Prlv*) 


2 


2 




2.9 


2.9 


2,8 


2.5 


1,8 


2.3 


Si zft of (litv oif Towii 


5 


5 




3.0 


3,0 


3,0 


7 S 




9 9 


13 ^ B - " 


22 


24 




2 2 


2,1 


2.4 




9 0 
4 ■ ^ 


9 n 


Annual Tuition ^ Feas*^ 


(10) 


(12) 




2.4 


2.1 


2.5 


2 4 




9 n 


Siza of Total EnfoTlTnent 


5 


5 




2 9 


3,0 


3.0 




9 


1 R 


Athletics Available 


31 


31 




2,9 


2,9 


2.9 


2.7 


2.3 


1,6 


Special Programs (pre- 


12 


8 




2 6 


2,5 


2.5 




9 1 
^ . ii 


1 A 


Adiais ^ "f nris Tnf otTnati 


16 


8 




7 7 


3.0 


2.8 




^ . O 


1 A 


Pjil^nfl^t* Plan 


9 


8 






2.9 


2.9 




1 Q 




Applicatloni Deadline 


5 


5 




3.0 


2.8 


3.0 


1.9 


2,0 


• 1,4 


Type of Institution 
(unlv.^ seminary^ etc*) 


9 


8 




2.9 


J.O 


2.9 


2.1 


1.5 


1,4 


Regional Accreditation 


J 


'J 




2.9 


3.0 


2.8 


2.2 


1,5 


1.4 


Religious Affiliation 








3.0 


3,0 


3.0 


2.5 


1,9 


1,3 


Emerging Fields* 


7 






2.8 


3,0 


2,9 


2.5 


2,2 


1.2 


Academic Characterls*- 
tlca of Students 


20 


22 




2,8 


3.0 


3.0 


2.6 


1.9 


il.2 


Campus Life (cultural^ 
social^ ecc»^ 


15 


15 






3.0 


2.6 


2.4 


1.9 


1.2 


Cmpus Activities (cho^ 
rus^ radio itation^ etc.) 


12 


12 




2.8 


3.0 


3.0 


2.7 


2.1 


1.0 


Faculty Qualifications 


2 


2 




2.8 


3.0 


2.8 


2.5 


2,3 


1.0 


Graduate Students* 


2 






2.4 


2.8 


2.8 


1.6 


1.8 


1.0 


Mean Rating Total 


600 


350 




2.8 


2.8 


2.7 


2.4 


2.2 


1,7 



*03L 4 only " ^ 

i^i^Each rating was on a 3-polnt scale (1-negatlva^ S^posltlve), Liaisons wara 
not asked to rate the ROTC category (COL 4 only; 4 characterlatics) , The cate- 
gory of "Costa" aubsumes "Annual Tuition and Fees" and "Annual Tuition^ Fees^ 
Room and Board," All means for these 2 aubheadlngs were Identical. 
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*6 

MEAN LIAISON RATINGS OF THE ... ,jACY OF THl GIS OCCUPATIONAL 
FILE, INCLUDING SEUCTOR AND DESCRIPODR CATEGORIES* 
(Listed in Rank Order of Overall Mean Rating) 



Category 


N Charac- 
teristics 


Variables 


Rated 




Complete- 


Rele- 


Clar- 


Logical- 


SeleotQrs 


ness 


vance 


ity 


ness 


15 Ocssupatlonal Clusters 


15 


2.5 


2.8 


2.8 


2.6 


Poraal Education 


14 


3.0 


3.0 


2.2 


2.5 


Special Vocational Training Tliae 


15 


2.7 


2.6 


1.9 


2.5 


Oeeupatlons Within Industries 


9 


2,5 


2.2 


2,1 


2.6 


Training Other Than Foraml 












Education 


9 


2.9 


2.2 


2.1 


2.2 


Aptitudes 


11 


1.9 


1.6 


1.8 


1.9 


Interests 


10 


1.5 


2.0 


1.4 


1.5 


Mean Selector Ratings 


83 


2.4 


2.3 


2.0 


2.3 


Descri-D tors 




15 Occupational Clusters 


15 


2.9 


2.7 


2.6 


2.6 


PnMnsI Rdiicatldon 


14 


3.0 


3,0 


2.0 


2.7 


nQctinHtioris Within Tnciiistrifis 


92 


2.3 


2.3 


2.6 


2.7 


Snpcial VocEtionBl Traintne Tims 


15 


2.6 


2.7 


1.6 


2.7 


Training Other Than Formal 












Education 


9 


2.9 


2.0 


1.9 


2.3 


Aptitudes 


11 


2.1 


1.7 


2.0 


2.3 


Interests 


10 


1.7 


1,8 


1.6 


1.7 


Physical Demands 


14 


2.9 


2.7 


3,0 


3.0 


Working Conditions 


21 


3.0 


2.5 


3,0 


3.0 


Work Activities 


12 


2.9 


2.5 


3.0 


3.0 


Other Qualifications (e.g^licens^ 


4 


2.4 


2,8 


2.7 


3.0 


Employment Outlook 


3 


1.7 


2,7 


2.6 


2.9 


Special Conditions (e.g., simmer 












only) 


3 


2.1 


2.3 


2.4 


2,9 


Areas of Work 


22 


2.1 


2.0 


2.8 


2.3 


Entry Level Earnings* 


13 


2.3 


2.0 


2.1 


2.7 


Mean Descriptor Ratings 


271 


2.4 


2.4 


2,4 


2.7 



*Each rating was on a 3-polnt scale (l=negatlve^ 3-positive). Mean 
ratlngi for the category of "Highest Level Earnings" were Identical to 
those for "Entry Level rmrnlngs*" 
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TABLE A7 

mm-mAR college fue-. frequency of use of each category in 

SEARClffiS, FEBRUARY 1974 (Results In Percentages) 



Categories Used* 


School 


All 


Total 


(In Rai^ Order) 


A 


B 


C 


D 


E 


Schools 


N 


Aeademie Progr^s of 
Study ^Majors") 


100% 




1 007 






98% 


265 


Geographic Location 


83 


92 




fin 


Qw 


81 


220 


Coaducation 


33 




o ^ 


77 


H-Q 


66 


178 


Competitiveness 


33 


•J Q 




% 1 

J L 




56 


177 


Si^e of Total Enrollment 


50 


0 


3 


54 


11 


24 


65 


Coiti 0.nc. Tuition ficFeea) 


0 


0 


2 


58 


5 


23 


63 


Admissions Information 


33 


8 


5 


33 


8 


17 


45 


Control (Public vs. 
Private) 


1 7 




A 


n 
u 




15 


40 


Financial Aid 


33 


A 








14 


38 


Sliie of City or Town 


n 


n 








14 


37 


Athletics Available 


17 


12 


11 


14 


7 


13 


36 


Special ProgrMs 


0 


0 


26 


2 


8 


11 


3X 


Tuition + Fees + Room & 
Board 


0 


0 


9 


4 


8 


6 


16 


Type of Institution 


17 


0 


6 


1 


16 


5 


14 


Religioui Affiliation 


17 


0 


3 


0 


24 


5 


13 


Campus Life 


0 


0 


9 


1 


0 


4 


10 


Special C^pus Activities 


0 


0 


5 


4 


0 


3 


9 


Emerging Fields 


0 


0 


0 


2 


0 


1 


2 


Academic Characteristics 
of Students 


17 


0 


0 


0 


0 


0 


1 


ROTC 


0 


0 


0 


1 


0 


0 


1 


Total Nwsber of Searches 
EKamlned 


6 


26 


100 


101 


37 




270 



^Within any one search^ multiple use of characteristics in a given cate- 
gory was counted only once. The categories of Graduate Students^ Ap- 
plication Deadline^ Regional Accreditation^ and Calendar Plan were not 
used at all, (The Faculty Qualifications category was Inadvertantly 
omitted from the counting^ but was little used.) 
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APPEOTIX B 



In this Appendix^ we will discuss the issue of student ability level 
in relation to the special assessment of the effects o£ GAG on vocabulary 
and decislon^making. We had originally hoped to compare honors and non^ 
honors students as one of the dimensions in our study of the effects of 
using the computer. However^ our sfflnple had no honors students in 
Schools B and and very few in School D*l 

AltDgether^ honors students constituted less than one^fourth of those 
taking either test. Because of the unequal Ns^ we could not use analysis 
of variance and thus could not compute the triple interaction of schools^ 
use of CAG^ and honors/nonhonors # Instead of analysis of variance^ we 
used a multiple regression progrffln ("Glypoth") to analyze significance of 
dif ferences#2 

When we inspected the cell means for Vocabulary and for Decision- 
Making j we found that wherever comparison with respect to honors status 
was possible (specifically^ where the cell N was more than 2)^ honors 
students^ whether users or non-^users^ performed better than the corres-^ 
ponding nonhonors students. But we also found that honors students at 
School A^ which was the poorest acadCTilcally^ perfomed more poorly than 
nonhonors students at other schools* Thus^ the honors designation sesns 
to indicate quite different levels of ability in different schools^ and 
Is not a suitable estimate of ability across schools. It wuld therefore 
not be appropriate to combine all honors students (or all honors users^ 
etc.) and compare than with all nonhonors students. 3 

The fact that the schools differed in ability level causes no problem 
in data analysis^ but whether ability was equally distributed OTong users 
and non-users Is not so clear. That honors students did better than com- 
parable nonhonors students is not surprising^ and does not by itsalf In^ 
terfere with analyzing effects of the use and schools' variables. The 
important question^ however^ is whether being an honors student gave users 
any extra advantage over non-users. As explained above^ we could not 
computer the triple analysis of variance interaction; however^ with the 
Glypoth progrffli^ we exmlned the use X ability interaction separately for 
School A and for School E,^ In both schools this Interaction was essen- 
tially zero on both tests^ suggesting that honors status gave no additional 
advantage to users. This result lends support to the procedure we adopted^ 
namely of combining honors and re^lar users^ and comparing them with 
honors plus regular non-usars^ without distinguishing between honors and 

^For eKample^ at School D: Vocabulary--of 86 students tested^ only 3 were 
honors users* Declslon-Making--of 69 students tested^ only 2 were honors nonusers 
2 

See footnote 1^ page 23, 

3 

Table A3 gives the overall means for all honors and nonhonors subgroups 
on the Vocabulary and Decision-Making tests^ but only for comparison pur* 
poses I the distortion caused by the unequal Ns is obvious. 
4 

In these two schools the cell Ns were all 9 or more. 
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nonhonors students. In further support of this procedure^ the liaisons 
reported that student ability was fairly homogeneous within the classes 
tested—a situation which would lead to the unequal Ns we obtained, And^ 
eonaiderlng that we tested In 22 English classes and 6 homerooms^ it seems 
reasonable to expect an even distribution o£ ability differences between 
the user and non-user groups, (Remember that the users and non-users 
being compared camm from the same classes.) We are willing to assume^ 
then^ that no Important initial abi lity differences existed bstween users 
and non^users that would account for the obtained results. 
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